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BACKGROUND INFORMATION 

The Maximus and Internet Growth 

As the Internet grows, the volume of data traversing the various networks that 
comprise the Internet has increased tremendously. While packet-based technology is 
widely believed to be the dominant technology going forward, in the short term, no single 
technology has emerged as the clear winner in the race to satisfy the seemingly insatiable 
demand for bandwidth, which is fueled by the spectacular growth of the Internet. 

At the same time, carriers respond to customers' demand for new applications by 
rolling out increasingly sophisticated data services, some of which are highly delay- 
sensitive and/or bandwidth-intensive (e.g., voice or video). The introduction of these 
differentiated services opens up opportunities for competing system vendors to introduce 



1 



7 



9 
10 
11 



nnn 4k A PATENT 

AZA-001/2001-P00l^ ^ 

1 new technologies and/or protocols. As a consequence, changes in the networks are 
characterized not only by vastly increased bandwidth requirements but also by a highly 
diverse array of technologies and protocols. 

The aforementioned changes impact the access, core/long-haul, and 
metro/regional networks in different ways. In the foreseeable future, the demand for 
6 high speed access (i.e., in the megabit to low gigabit range) appears to be satisfied by a 
multitude of competing technologies, e.g., xDSL, cable modem, Frame Relay, Tl, T3, 
8 ATM, and Gigabit Ethernet. Some of the newer technologies such as xDSL and cable 
modem have made impressive gains, enabling users to transmit and receive data at vastly 
increased data rates, thereby facilitating higher-bandwidth applications, some which were 
impractical only a few years ago when dial-up or lower speed ISDN connections were the 

12 dominant access technologies. 

1 3 Corresponding changes also occur in the core/long-haul networks. In the core 
network, the emergence of an all-optical core and DWDM technologies have made it 
practical to transfer terabits of data across long-haul trunks at substantially lower costs. 
However, all-optical technologies have not progressed to the point where it is possible to 

17 process packets and/or cells in the optical domain. Processing intelligence is still very 

18 much the province of electronics. Accordingly, an all-optical core, when operated in its 

19 pure optical mode, is substantially devoid of processing intelligence, such as the type 

20 required to correctly perform end-to-end routing of a packet or cell from one user to 

21 another. Without resorting to the electrical plane, it is currently not practical to offer 

22 differentiated data services (e.g., routing different streams of traffic in different ways in 

23 accordance to their classes of service) via all-optical technologies. 

24 The metro/regional network bears the full brunt of the powerful changing forces 

25 that affect the access and core networks. As an aggregation mechanism for access 

26 networks, the metro/regional network must move to higher speeds to cope with the 

27 increased bandwidth requirements. The metro/regional network must also interoperate 

28 with legacy and emerging transmission technologies and protocols, as well as adapt the 

29 divergent technologies and protocols to its own transmission requirements and the 

30 requirements of the core network. The metro/regional network must also perform 

3 1 extensive cell and packet processing, as these tasks are no longer performed by the 
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1 optical core network. To facilitate differentiated service offerings, the metro/regional 

2 network must also handle traffic management for different data streams having different 

3 classes of service. In short, the metro/regional network of tomorrow is characterized by 

4 higher speeds, diverse transmission technologies and protocols, and enhanced processing 

5 requirements. 
6 

7 Capabilities of the Maximus 

8 The Maximus is designed to capitalize on these trends. It is a high integration 

9 ASIC designed for high speed switching and routing systems. The Maximus also finds 

10 applications in access devices operating at OC-192, or as an aggregation device in the 

1 1 core network. In a typical line card deployment, the Maximus is implemented between 

12 an OC-192 or a quad OC-48 multi-service framer and the switch fabric. 

2 13 

Nj 14 BRIEF DESCRIPTION OF THE DRAWINGS 

J 1 5 The present invention is illustrated by way of example, and not by way of 

W 1 6 limitation, in the Figures of the accompanying drawings and in which like reference 

M= 17 numerals refer to similar elements and in which: 
q 18 

2 19 Figure 1 is a figure showing multi-service system.. 

i 20 Figure 2 is a figure showing OC-192 line card for traffic manager application^ 

rf 21 Figure 3 is a figure showing OC-192 line card for traffic manager application. 

22 Figure 4 is a figure showing various applications of the maximus chipset along with the 

23 type field 

24 Figure 5 is a figure showing traffic manager applications. 

25 Figure 5 A is a figure showing uses of the Maximus chip with cell-based switch fabric. 

26 Figure 6 is a figure showing SAR applications, 

27 Figure 6 A is a figure showing uses of the Maximus chip with packet-b ased switch fabric. 

28 Figure 7 is a figure showing ingress tm (ATM=>ATM), 

29 Figure 8 is a figure showing ingress tm (ATM=> MPLS packet), 

30 Figure 9 is a figure showing ingress tm (MPLS packet=>ATM), 

31 Figure 10 is a figure showing ingress tm (packet=>packet). 
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1 Figure 1 1 is a figure showing ATM encapsulation^ 

2 Figure 12 is a figure showing reassembly. 

3 Figure 13 is a figure showing ingress packet bypass. 

4 Figure 14 is a figure showing egress tm (ATM=>ATM). 

5 Figure 15 is a figure showing egress tm (ATM=> MPLS packet). 

6 Figure 16 is a figure showing egress tm (MPLS packet =>ATM). 

7 Figure 17 is a figure showing egress tm (packet=>packet) i 

8 Figure 18 is a figure showing ATM de-encapsulation,. 

9 Figure 19 is a figure showing segmentation. 

10 Figure 20 is a figure showing egress packet bypass^ 

1 1 Figure 21 is a figure showing Maximus chipset top level block diagram. 

12 Figure 22 is a figure showing signals^ 

Q 13 Figure 23 is a figure showing block addresses. 

J? ; 14 Figure 24 is a figure showing read access and timings definitions^ 

^ 15 Figure 25 is a figure showing read access characteristic parameters,, 

14 16 Figure 26 is a figure showing write access and timings definitions,, 

r; 17 Figure 27 is a figure showing write access characterisitcs parameters. 

3 

s 1 8 Figure 28 is a figure showing interface from the CPU if to the blocks. 

fp 19 Figure 29 is a figure showing global sync. 

^ 20 Figure 30 is a figure showing slot_count and CPU_port signals. 

Q 21 Figure 3 1 is a figure showing interface of the CPU if block. 

^ 22 Figure 32 is a figure showing internal implementation for locking the current CPU 

23 access, parameters. 

24 Figure 33 is a figure showing test mux block diagram. 

25 Figure 34 is a figure showing soft resets^ 

26 Figure 35 is a figure showing input/output disable,, 

27 Figure 36 is a figure showing registers. 

28 Figure 37 is a figure showing incoming spi-4 interface. 

29 Figure 38 is a figure showing input control block in spii^ 

30 Figure 39 is a figure showing output control block in spii. 

3 1 Figure 40 is a figure showing bit align blocks 
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1 Figure 41 is a figure showing drop idle state machine.. 

2 Figure 42 is a figure showing control word format. 

3 Figure 43 is a figure showing descriptions of fields in the control words. 

4 Figure 44 is a figure showing control word type list. 

5 Figure 45 is a figure showing rx state machine transition table^ 

6 Figure 46 is a figure showing extent of dip-4 codewords^ 

7 Figure 47 is a figure showing example of dip-4 encoding (odd parity). 

8 Figure 48 is a figure showing example of a port calendar. 

9 Figure 49 is a figure showing fifo status state machine. 

10 Figure 50 is a figure showing example of dip-2 encoding (odd parity). 

1 1 Figure 51 is a figure showing summary of internal memories. 

12 Figure 52 is a figure showing interface ports. 

□ 13 Figure 53 is a figure showing register description. 

14 Figure 54 is a figure showing outgoing spi-4 interface. 

Nl 15 Figure 55 is a figure showing training sequence. 

^ 16 Figure 56 is a figure showing tx state machine transition table^ 

17 Figure 57 is a figure showing extent of dip-4 codewords., 

5 1 8 Figure 58 is a figure showing example of dip-4 encoding (odd parity). 

Pi 

m 19 Figure 59 is a figure showing example of transfer data byte ordering^ 

D 20 Figure 60 is a figure showing tx state machine block. 

□ 21 Figure 61 is a figure showing port calendar. 

22 Figure 62 is a figure showing status interface to database. 

23 Figure 63 skew detection,, 

24 Figure 64 is a figure showing seg_table - 64 x 26 bits internal dual-port SRAM. 

25 Figure 65 is a figure showing interface. 

26 Figure 66 is a figure showing timing diagram-reassembly port empty. 

27 Figure 67 is a figure showing timing diagram-back-to-back port transfer. 

28 Figure 68 is a figure showing register description. 

29 Figure 69 is a figure showing software interface registers. 

30 Figure 70 is a figure showing read outgoing spi start-up parameters. 

31 Figure 71 is a figure showing write outgoing spi start-up parameters. 
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1 Figure 72 is a figure showing read outgoing spi start-up parameters, 

2 Figure 73 is a figure showing write outgoing spi start-up parameters, 

3 Figure 74 is a figure showing read outgoing spi start-up parameters^ 

4 Figure 75 is a figure showing write outgoing spi start-up parameters, 

5 Figure 76 is a figure showing write port parameters after initialization, 

6 Figure 77 is a figure showing read port parameters after initialization^ 

7 Figure 78 is a figure showing read port full parameters, 

8 Figure 79 is a figure showing write port full parameters. 

9 Figure 80 is a figure showing read control status register 

10 Figure 81 is a figure showing write spio_tstmux_sel, 

1 1 Figure 82 is a figure showing read spio_tstmux_sel, 

12 Figure 83 is a figure showing lookup engine and neighboring blocks, 

13 Figure 84 is a figure showing top level block diagram of the lookup engine, 

14 Figure 85 is a figure showing data dependency on sop cell followed none sop cell on the 

15 same port 

16 Figure 86 is a figure showing header selection. 

17 Figure 87 is a figure showing label hashing & record retrieval 

1 8 Figure 88 is a figure showing port parameters, it is 64 entries. 

19 Figure 89 is a figure showing encap table structure, it is 8 entries, 

20 Figure 90 is a figure showing port type. 

21 Figure 91 is a figure showing record memory, it is 2meg entries per memory, 

22 Figure 92 is a figure showing parameter width, 

23 Figure 93 is a figure showing external interface signals, 

24 Figure 94 is a figure showing external interface signal names and direction. 

25 Figure 95 is a figure showing global signals, 

26 Figure 96 is a figure showing CPU interface to look up engine, 

27 Figure 97 is a figure showing memory size. 

28 Figure 98 is a figure showing memory size bits, 

29 Figure 99 is a figure showing memory chip select. 

30 Figure 100 is a figure showing memory chip select values, 

31 Figure 101 is a figure showing address space. 
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1 Figure 102 is a figure showing registers addresses. 

2 Figure 103 is a figure showing nop command, 

3 Figure 104 is a figure showing read lut memory 1 command. 

4 Figure 105 is a figure showing write lut memory 1 command, 

5 Figure 106 is a figure showing read lut memory2 command. 

6 Figure 107 is a figure showing write lut memory2 command^ 

7 Figure 108 is a figure showing read port memory command. 

8 Figure 109 is a figure showing write port memory command. 

9 Figure 1 10 is a figure showing setup connection command for key command^ 

10 Figure 1 1 1 is a figure showing teardown connection command for key command, 

1 1 Figure 1 12 is a figure showing get hash FID command, 

12 Figure 1 13 is a figure showing get hash FED response, 

□ 13 Figure 1 14 is a figure showing read cam memory 1 command, 

ci 14 Figure 1 1 5 is a figure showing write cam memory 1, 

Nl 1 5 Figure 1 16 is a figure showing read cam memory2, 

j3 16 Figure 1 17 is a figure showing write cam memory 2. 

17 Figure 1 18 is a figure showing init all memories, 

3 18 Figure 1 19 is a figure showing init memory 1 & cam 1, 

m 19 Figure 120 is a figure showing init memory 2 & cam 2, 

Q 20 Figure 121 is a figure showing ATM types, 

G 21 Figure 122 is a figure showing ATM MPLS tag location & format, 

22 Figure 123 is a figure showing MPLS format, 

23 Figure 124 is a figure showing lookup types, 

24 Figure 125 is a figure showing header format, 

25 Figure 126 is a figure showing data position within the header format, 

26 Figure 127 is a figure showing header substitution, 

27 Figure 128 is a figure showing special header structure, 

28 Figure 129 is a figure showing encap table structure, 

29 Figure 1 30 is a figure showing the encapsulation header replacement in 12 case, 

30 Figure 131 is a figure showing hashing and memory accesses, 

31 Figure 132 is a figure showing hash timing and lookup operations. 
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Figure 133 is a figure showing types of segmentation and applications for various traffic 



1 

2 types. 



3 Figure 1 34 is a figure showing segmentation engine and neighboring blocks, 

4 Figure 1 35 is a figure showing spi-4 data flow, 

5 Figure 1 36 is a figure showing type 1 segmentation, 

6 Figure 137 is a figure showing type 2 segmentation, 

7 Figure 138 is a figure showing type 3 segmentation, 

8 Figure 1 39 is a figure showing type 4 segmentation, 

9 Figure 140 is a figure showing status cell access flow, 

10 Figure 141 is a figure showing CPU access flow, 

1 1 Figure 142 is a figure showing top-level block diagram for segmentation engine, 

12 Figure 143 is a figure showing error conditions, 

1 3 Figure 1 44 is a figure showing summary of internal memories, 

14 Figure 145 is a figure showing segjable - 65 x 39 bits internal dual-port SRAM 

1 5 Figure 146 is a figure showing free_buffer - 80 x 7 bits internal SRAM, 

16 Figure 147 is a figure showing q_fifo - 80 x 70 bits internal SRAM, 

17 Figure 148 is a figure showing data SRAM - 640 x 64 bits internal SRAM, 

18 Figure 149 is a figure showing crcjable - 65 x 32 bits internal dual-port SRAM, 

19 Figure 1 50 is a figure showing statistic SRAM - 65 x 80 bits internal SRAM, 

20 Figure 1 5 1 is a figure showing system, 

: 21 Figure 152 is a figure showing segmentation O lookup, 

22 Figure 1 53 is a figure showing segmentation O memory manager, 

23 Figure 154 is a figure showing segmentation O PFQ, 

24 Figure 155 is a figure showing segmentation O sch, 

25 Figure 156 is a figure showing segmentation CPU interface, 

26 Figure 1 57 is a figure showing timing diagrams, 

27 Figure 1 58 is a figure showing memory access, 

28 Figure 1 59 is a figure showing registers, 

29 Figure 1 60 is a figure showing summary of commands, 

30 Figure 161 is a figure showing CPU inject packet, 

3 1 Figure 162 is a figure showing read seg fbl memory. 
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1 Figure 163 is a figure showing write seg fbl memory, 

2 Figure 164 is a figure showing read seg crc memory, 

3 Figure 165 is a figure showing write seg crc memory, 

4 Figure 166 is a figure showing read seg statistic memory, 

5 Figure 167 is a figure showing write seg statistic memory, 

6 Figure 168 is a figure showing read seg queue memory, 

7 Figure 169 is a figure showing write seg queue memory, 

8 Figure 170 is a figure showing read seg data memory, 

9 Figure 1 7 1 is a figure showing write seg x data memory, 

10 Figure 1 72 is a figure showing read seg tbl memory, 

11 Figure 1 73 is a figure showing write seg tbl memory, 

12 Figure 174 is a figure showing memory manager block diagram, 

13 Figure 175 is a figure showing internal memory manager block diagram, 

14 Figure 176 is a figure showing multicast and BID/imBID release, 

15 Figure 177 is a figure showing enqueue process in the internal memory manager, 

16 Figure 178 is a figure showing dequeue process in the internal memory manager, 

17 Figure 179 is a figure showing SDRAM manager block diagram, 

18 Figure 180 is a figure showing memory controller configuration, 

19 Figure 181 is a figure showing summary of memories, 

20 Figure 1 82 is a figure showing summary of CAM, 

2 1 Figure 1 83 is a figure showing segmentation engine, 

22 Figure 1 84 is a figure showing per flow queue engine, 

23 Figure 1 85 is a figure showing DDR-SDSRAM, 

24 Figure 1 86 is a figure showing reassembly, 

25 Figure 1 87 is a figure showing CPU interface, 

26 Figure 1 88 is a figure showing global sync, 

27 Figure 189 is a figure showing memory size is a figure showing 64 mbytes, 

28 Figure 190 is a figure showing memory size is a figure showing 256 mbytes, 

29 Figure 191 is a figure showing register description, 

30 Figure 192 is a figure showing CPU_wr_SDRAM, 

31 Figure 193 is a figure showing CPU_rd_SDRAM, 
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1 Figure 194 is a figure showing CPU_wr_ fiml, 

2 Figure 195 is a figure showing CPU_rd_fimL 

3 Figure 1 96 is a figure showing CPU_wr_cam, 

4 Figure 1 97 is a figure showing CPU_rd_canu 

5 Figure 1 98 is a figure showing CPU Jnitjnm, 

6 Figure 1 99 is a figure showing inputs from segmentation. 

7 Figure 200 is a figure showing interface with SDRAM bw optimizer is a figure showing 

8 SWM requests to write to SDRAM, 

9 Figure 201 is a figure showing interface to SDRAM bw optimizer is a figure showing 

10 write to SDRAM. 

1 1 Figure 202 is a figure showing interface with per flow queue is a figure showing receive 

12 de-queue from PFQ, 

13 Figure 203 is a figure showing interface with SDRAM bw optimizer is a figure showing 

14 SRM requests to read from SDRAM. 

1 5 Figure 204 is a figure showing interface with SDRAM bw optimizer is a figure showing 

16 SRM read from SDRAM, 

1 7 Figure 205 is a figure showing interface with reassembly, 

1 8 Figure 206 is a figure showing applications for various traffic types, 

19 Figure 207 is a figure showing reassembly engine and neighboring blocks, 

20 Figure 208 is a figure showing type 1 reassembly (ingress memory cells => switch cells), 

2 1 Figure 209 is a figure showing type 1 reassembly (aal5 memory cells => switch cells), 

22 Figure 2 10 is a figure showing type 1 reassembly (aal5 like memory cells => switch 

23 cells), 

24 Figure 21 1 is a figure showing type 2 reassembly (52 byte ATM cells => packet), 

25 Figure 212 is a figure showing type 3 reassembly (48 byte ATM cells => packet), 

26 Figure 2 1 3 is a figure showing type 4 reassembly (switch cells => packet), 

27 Figure 214 is a figure showing type 5 reassembly (52 byte ATM cells => ATM cells with 

28 translation), 

29 Figure 21 5 is a figure showing type 6 reassembly (48 byte ATM cells => packet), 

30 Figure 2 1 6 is a figure showing type 7 reassembly (48 byte cells => AAL-5 cells), 

3 1 Figure 2 1 7 is a figure showing type 8 reassembly (switch cells => packet). 



10 



AZA-001/2001-P00' 



# 



PATENT 





1 




2 




3 




4 




5 




6 




7 




8 




9 




10 




11 




12 


Q 

est 


13 




14 




15 




16 


Ls. 


17 


5 


18 


JSC 


19 




20 


Li 


21 




22 




23 




24 




25 




26 




27 




28 




29 




30 




31 



Figure 218 is a figure showing top level block diagram for the reassembly engine. 

Figure 219 is a figure showing free buffer list block diagram. 

Figure 220 is a figure showing crc_gen block diagram. 

Figure 221 is a figure showing ATM MPLS tag location & format. 

Figure 222 is a figure showing MPLS format 

Figure 223 is a figure showing ATM types. 

Figure 224 is a figure showing seg_table - 64 x 18 bits internal single-port SRAM. 

Figure 225 is a figure showing output_control_memory - 128 x 18 bit internal SRAJVL 

Figure 226 is a figure showing output_port_queue - 64 x 18 bit internal SRAJVL 

Figure 227 is a figure showing free_buffer- 128 x 7 bit internal SRAM. 

Figure 228 is a figure showing datajSRAM - 1024 x 64 bit internal dual port SRAM^ 

Figure 229 is a figure showing header_SRAM - 256 x 64 bit internal dual port SRAM^ 

Figure 230 is a figure showing CPU_data_SRAM - 36 x 64 bit internal dual port SRAJVL 

Figure 231 is a figure showing crc_table - 64 x 32 bits internal dual-port SRAM. 

Figure 232 is a figure showing statistic SRAM - 64 x 1 12 bit internal SRAJVL 

Figure 233 is a figure showing look-up SRAM - 4m x 36 bit external SRAM. 

Figure 234 is a figure showing flow header format. 

Figure 235 is a figure showing control word formats^ 

Figure 236 is a figure showing Maximus header format 

Figure 237 is a figure showing ATM header format. 

Figure 238 is a figure showing MPLS format. 

Figure 239 is a figure showing aal5 trailer format 

Figure 240 is a figure showing system.. 

Figure 241 is a figure showing reassembly <=> memory manager. 

Figure 242 is a figure showing reassembly <=> SPL 

Figure 243 is a figure showing reassembly OCPU. 

Figure 244 is a figure showing internal memory manager to reassembly. 

Figure 245 is a figure showing reassembly to spi out - port empty and back to back. 

Figure 246 is a figure showing timing diagram-back-to-back port transfer 

Figure 247 is a figure showing memories. 

Figure 248 is a figure showing error conditions. 
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1 Figure 249 is a figure showing registers. 

2 Figure 250 is a figure showing commands. 

3 Figure 251 is a figure showing read external ras memory, 

4 Figure 252 is a figure showing write external ras memory. 

5 Figure 253 is a figure showing flow control format. 

6 Figure 254 is a figure showing read internal data memory. 

7 Figure 255 is a figure showing write internal data memory. 

8 Figure 256 is a figure showing read four internal memories (partial crcl, partial crc2, 

9 reassembly table, output queue). 

10 Figure 257 is a figure showing write four internal memories (partial crcl, partial crc2, 

1 1 reassembly table, output queue). 

12 Figure 258 is a figure showing read port statistics. 

13 Figure 259 is a figure showing write port statistics. 

^ 1 4 Figure 260 is a figure showing read CPU data memory^ 

m 1 5 Figure 26 1 is a figure showing write CPU data memory, 

% 16 Figure 262 is a figure showing read 2 internal memories (free buffer list (fbl), output 
^ 17 control). 

3 

□ 18 Figure 263 is a figure showing write 2 internal memories (free buffer list (fbl), output 

m 

£ 19 control). 

M* 20 Figure 264 is a figure showing read internal header data memory. 

Ct 21 Figure 265 is a figure showing write internal header data memory. 

22 Figure 266 is a figure showing get CPU cell command. 

23 Figure 267 is a figure showing initialize free buffer list and output port queue memories 

24 command. 

25 Figure 268 is a figure showing summary of lookup engine block tags for various traffic 

26 types. 

27 Figure 269 is a figure showing ATM header format. 

28 Figure 270 is a figure showing MPLS label format. 

29 Figure 271 is a figure showing ethernet frame format (IEEE 802.3 format^ 

30 Figure 272 is a figure showing EP (internet protocol) header 
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Figure 273 is a figure showing frame relay (fr) layer 2 frame format (shown with IP 




2 


payoload). 




3 


Figure 274 is a figure showing PPP full frame format for unnumbered mode operation. 




4 


Figure 275 is a figure showing flow control entry format. 




5 


Figure 276 is a figure showing tm to fabric flow control summary. 




6 


Figure 277 is a figure showing fabric to tm control summary.. 




7 


Figure 278 is a figure showing internal block diagram. 




8 


Figure 279 is a figure showing graphic representation of an enqueue and dequeue FID. 




9 


Figure 280 is a figure showing operation sequence table^ 




10 


Figure 281 is a figure showing enqueue block diagram^ 




11 


Figure 282 is a figure showing dequeue block digram. 




12 


Figure 283 is a figure showing free buffer manager block diagram^ 




13 


Figure 284 is a figure showing random number generator. 
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Figure 285 is a figure showing memory size,. 




15 


Figure 286 is a figure showing memory chip select 




16 


Figure 287 is a figure showing data dependency^ 


3 


17 


Figure 288 is a figure showing RED memoi^ 




18 


Figure 289 is a figure showing RED memory byte enable., 
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Figure 290 is a figure showing statistics counter location 0. 




20 


Figure 291 is a figure showing statistics counter location 1. 




21 


Figure 292 is a figure showing statistics counter byte enable.. 




22 


Figure 293 is a figure showing dequeue memory. 




23 


Figure 294 is a figure showing dequeue memory byte enable^ 




24 


Figure 295 is a figure showing FED enqueue location 0^ 




25 


Figure 296 is a figure showing FID enqueue memory location L 




26 


Figure 297 is a figure showing FID enqueue memory byte enable^ 




27 


Figure 298 is a figure showing BID external data structure^ 




28 


Figure 299 is a figure showing BID external data structure byte enable,. 




29 


Figure 300 is a figure showing dequeue memory. 




30 


Figure 301 is a figure showing dequeue memory byte enabtei 




31 


Figure 302 is a figure showing multicasting FID enqueue memory location 0. 
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1 Figure 303 is a figure showing multicasting FID enqueue memory location 1 , 

2 Figure 304 is a figure showing multicasting FID enqueue memory byte enable. 

3 Figure 305 is a figure showing multicasting FID root memory, 

4 Figure 306 is a figure showing FID dequeue leaf tunneling internal data structure, 

5 Figure 307 is a figure showing FID enqueue leaf tunneling internal data structure location 

6 0, 

7 Figure 308 is a figure showing FED enqueue leaf tunneling internal data structure location 

8 1. 

9 Figure 309 is a figure showing FID enqueue leaf tunneling internal data structure byte 

10 enable. 

1 1 Figure 3 1 0 is a figure showing tunneling FID root data structure. 

12 Figure 3 1 1 is a figure showing interface signals, 

0 1 3 Figure 3 1 2 is a figure showing CPU interface to PFQ, 

SJ 14 Figure 3 1 3 is a figure showing segmentation interface diagram. 

^ 1 5 Figure 3 1 4 is a figure showing segmentation interface. 

Ul 1 6 Figure 3 1 5 is a figure showing DBS & CBWFQ interface, 

U 1 7 Figure 3 1 6 is a figure showing DBS interface, 

^ 1 8 Figure 3 1 7 is a figure showing memory and PFQ interface, 

01 19 Figure 3 1 8 is a figure showing memory and PFQ interface signals, 

2 20 Figure 3 1 9 is a figure showing memory manager commands to per flow queue, 
O 21 Figure 320 is a figure showing per flow queue commands to memory manager, 

22 Figure 321 is a figure showing mem to PFQ commands, 

23 Figure 322 is a figure showing PFQ to mem commands, 

24 Figure 323 is a figure showing memory manager and the per flow queue interface, 

25 Figure 324 is a figure showing PFQ and memory manager PFQ command structure, 

26 Figure 325 is a figure showing PFQ to segmentation timing diagram, 

27 Figure 326 is a figure showing address space, 

28 Figure 327 is a figure showing registers space, 

29 Figure 328 is a figure showing nop command, 

30 Figure 329 is a figure showing read FID enqueue memory command, 

31 Figure 330 is a figure showing write FID enqueue memory command. 
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1 Figure 33 1 is a figure showing read FID dequeue memory command^ 

2 Figure 332 is a figure showing write FID dequeue memory command^ 

3 Figure 333 is a figure showing read BED memory commands 

4 Figure 334 is a figure showing write BID memory command. 

5 Figure 335 is a figure showing read stat memory command. 

6 Figure 336 is a figure showing write stat memory command^ 

7 Figure 337 is a figure showing setup connection command. 

8 Figure 338 is a figure showing tear-down connection command. 

9 Figure 339 is a figure showing init FID memories. 

10 Figure 340 is a figure showing init BID memory. 

1 1 Figure 341 is a figure showing timing that includes the FID enqueue, dequeue, BID and 

12 statistics memories. 

Q 1 3 Figure 342 is a figure showing enqueue memory^ 

\j 14 Figure 343 is a figure showing RED memory^ 

^ 1 5 Figure 345 is a figure showing dequeue memory. 

yj 16 Figure 346 is a figure showing BID memory. 

yf 17 Figure 347 is a figure showing statistics memory^ 

;L 1 8 Figure 348 is a figure showing basic block diagram. 

rp 19 Figure 349 is a figure showing interface to/from shaper. 

ff 20 Figure 350 is a figure showing general signals^ 

O 21 Figure 351 is a figure showing status interface from output spi. 

22 Figure 352 is a figure showing flowid memory. 

23 Figure 353 is a figure showing pkt memory. 

24 Figure 354 is a figure showing port calendar memory. 

25 Figure 355 is a figure showing shp_strict memoy^ 

26 Figure 356 is a figure showing shp_out_port memory. 

27 Figure 357 is a figure showing register descriptions^ 

28 Figure 358 is a figure showing read external FID memory. 

29 Figure 359 is a figure showing write external FID memory. 

30 Figure 360 is a figure showing setup FID connection. 

3 1 Figure 361 is a figure showing read external pkt memory. 

15 
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1 Figure 362 is a figure showing write external pkt memory^ 

2 Figure 363 is a figure showing read port calendar memory. 

3 Figure 364 is a figure showing write port calendar memory. 

4 Figure 365 is a figure showing read shp strict memory^ 

5 Figure 366 is a figure showing write shp strict memory. 

6 Figure 367 is a figure showing read shp out memory. 

7 Figure 368 is a figure showing write shp memory^ 

8 Figure 369 is a figure showing init shp_out_port memory. 

9 Figure 370 is a figure showing init shp_strict memory. 

10 Figure 371 is a figure showing timing diagram. 

1 1 Figure 372 is a figure showing shaper. 

12 Figure 373 is a figure showing scheduler 
G 13 Figure 374 is a figure showing PFQ. 

J 14 Figure 375 is a figure showing ingress/egress mode of operations^ 

"2 15 Figure 376 is a figure showing interface to/from scheduler, 

bj 16 Figure 377 is a figure showing scheduler's interface. 

U 17 Figure 378 is a figure showing signal description^ 

s 18 Figure 379 is a figure showing simple round-robin algorithm^ 

ri 19 Figure 380 is a figure showing weighted-round-robin scheduling^ 

ff 20 Figure 381 is a figure showing creating smaller bursts with factor 

S 21 Figure 382 is a figure showing strict priority^ 

22 Figure 383 is a figure showing links between all levels of scheduling^ 

23 Figure 384 is a figure showing links between memories during input phase (the ingress 

24 modeX 

25 Figure 385 is a figure showing links between memories during output phase (the egress 

26 mode). 

27 Figure 386 is a figure showing links between memories during output phase (the ingress 

28 modeX 

29 Figure 387 is a figure showing external memory - next flowid^ 

30 Figure 388 is a figure showing internal memory - qos parameters^ 

3 1 Figure 389 is a figure showing internal memory - qos descriptors. 

16 
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1 Figure 390 is a figure showing internal memory - port parameters. 

2 Figure 391 is a figure showing internal memory - port descriptor 

3 Figure 392 is a figure showing internal memory - next port 

4 Figure 393 is a figure showing registers^ 

5 Figure 394 is a figure showing read FID next memory. 

6 Figure 395 is a figure showing write FID next memory^ 

7 Figure 396 is a figure showing read qos parameters^ 

8 Figure 397 is a figure showing write qos parameters. 

9 Figure 398 is a figure showing read qos descriptor. 

10 Figure 399 is a figure showing write qos descriptor 

1 1 Figure 400 is a figure showing read port parameters. 

12 Figure 401 is a figure showing write port parameters^ 
□ 1 3 Figure 402 is a figure showing read port descriptor 
Q 14 Figure 403 is a figure showing write port descriptor. 
U 15 Figure 404 is a figure showing read port next. 

yj 1 6 Figure 405 is a figure showing write port next 

rf 17 Figure 406 is a figure showing init qos. 

^ 18 Figure 407 is a figure showing init port. 

0§ 19 Figure 408 is a figure showing global timing for input/output stages^ 

rf 20 Figure 409 is a figure showing the input phase pipelines^ 

Q 21 Figure 410 is a figure showing the output phase pipelines,, 

22 Figure 41 1 is a figure showing shaper/meter block interface. 

23 Figure 412 is a figure showing traffic shaper/meter interface for dual-leaky-bucket 

24 Figure 413 is a figure showing traffic shaper/meter interface for single-leaky-bucket 

25 Figure 414 is a figure showing general signals. 

26 Figure 41 5 is a figure showing progress of a leaky bucket algorithm. 

27 Figure 416 is a figure showing links in the timing wheel 

28 Figure 417 is a figure showing example for shaping FID in the timing wheel 

29 Figure 418 is a figure showing calculating future time slot 

30 Figure 419 is a figure showing the usage of peak and sustained rates. 

31 Figure 420 is a figure showing Rin and K factor. 

17 
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1 


Figure 421 i 


s a figure showing credit calculation diagram. 




2 


Figure 422 i 


s a figure showing calculation of threshold meter 




3 


Figure 423 i 


s a figure showing memory description. 




4 


Figure 424 i 


s a figure showing shaper rate ID for shaper mode^ 




5 


Figure 425 i 


s a figure showing meter threshold for meter mode. 




6 


Figure 426 i 


s a figure showing shaper slot memory. 




7 


Figure 427 i 


s a figure showing FID meml for shapen 




8 


Figure 428 i 


s a figure showing FID meml for meter. 




9 


Figure 429 i 


s a figure showing FED mem 2 for shaper. 




10 


Figure 430 i 


s a figure showing FED mem2 for meter. 




11 


Figure 431 i 


s a figure showing register description. 




12 


Figure 432 i 


s a figure showing read FID1 memory. 




13 


Figure 433 i 


is a figure showing write FID1 memory when using shaper. 




14 


Figure 434 i 


is a figure showing read FID2 memory. 


. *~ 


15 


Figure 435 i 


is a figure showing write FID2 memory. 


i s it 


16 


Figure 436 i 


is a figure showing read shpslot memory. 




17 


Figure 437 i 


is a figure showing write shp slot memory. 




18 


Figure 438 


is a figure showing read rate_id memory. 




19 


Figure 439 


is a figure showing write rate_id memory^ 




20 


Figure 440 


is a figure showing setup connection command. 




21 


Figure 441 


is a figure showing read FED1 memory. 




22 


Figure 442 


is a figure showing write FID1 memory. 




23 


Figure 443 


is a figure showing read FED2 memory. 




24 


Figure 444 


is a figure showing write FID2 memory^ 




25 


Figure 445 


is a figure showing read threshold (internal) memory. 




26 


Figure 446 


is a figure showing write threshold (internal) memory. 




27 


Figure 447 


is a figure showing read modwrite meter. 




28 


Figure 448 


is a figure showing setup connection command. 




29 


Figure 449 


is a figure showing timing diagrams for meter. 




30 


Figure 450 


is a figure showing block addresses^ 




31 


Figure 45 1 


is a figure showing read access and timings definitions^ 
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1 Figure 452 is a figure showing read access timing and definitions table, 

2 Figure 453 is a figure showing write access and timings definitions. 

3 Figure 454 is a figure showing write access and timings definition table. 

4 Figure 455 is a figure showing interface from the CPU IF to the blocks. 

5 Figure 456 is a figure showing configuration modes. 

6 Figure 457 is a figure showing global sync, 

7 Figure 458 is a figure showing slot_count and CPU_port signals. 

8 Figure 459 is a figure showing multiplexing the test busses from the blocks, 

9 Figure 460 is a figure showing interface of the CPU IF blocks 

10 Figure 461 is a figure showing internal implementation for locking the current CPU 

1 1 access parameters. 

1 2 Figure 462 is a figure showing register description, 
y 1 3 Figure 463 is a figure showing CPU interface signals, 
H 14 Figure 464 is a figure showing block addresses, 

J 1 5 Figure 465 is a figure showing CPUif block register 

!ff 1 6 Figure 466 is a figure showing PFQ block register 

M= 17 Figure 467 is a figure showing read FID enqueue memory (72 bits) - up to 2m locations, 

^ 1 8 Figure 468 is a figure showing write FID enqueue memory (72 bits) - up to 2m locations, 

fj 19 Figure 469 is a figure showing read FID dequeue memory (36 bits)- up to lm locations, 

U 20 Figure 470 is a figure showing write FID dequeue memory (36 bits) - up to lm 

Gt 

j~ 21 locations, 

22 Figure 471 is a figure showing read BID dequeue memory (36 bits) - up to 16m 

23 locations^ 

24 Figure 472 is a figure showing write BID memory (36 bits) - up to 16m locations, 

25 Figure 473 is a figure showing read stat memory (72 bits) - up to 2m locations, 

26 Figure 474 is a figure showing write stat memory (72 bits) - up to 2m locations, 

27 Figure 475 is a figure showing setup connection command for FID. 

28 Figure 476 is a figure showing tear-down connection command for FID, 

29 Figure 477 is a figure showing init FID memories, 

30 Figure 478 is a figure showing init BID memory, 

3 1 Figure 479 is a figure showing database block registers, 

19 
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Figure 480 is a figure showing read external FID memory 




2 


Figure 481 is a figure showing write external FID memory. 




3 


Figure 482 is a figure showing setup FID connection. 




4 


Figure 483 is a figure showing read external pkt memory ; 




5 


Figure 484 is a figure showing write external pkt memory^ 




6 


Figure 485 is a figure showing read port calendar memoir 




7 


Figure 486 is a figure showing write port calendar memory^ 




8 


Figure 487 is a figure showing read shp strict memo^ 




9 


Figure 488 is a figure showing write shp strict memory. 




10 


Figure 489 is a figure showing read shp out memoir 




11 


Figure 490 is a figure showing write shp out memory. 




12 


Figure 491 is a figure showing init shp_out_port memory. 




13 


Figure 492 is a figure showing init shp_strict memory. 


H 


14 


Figure 493 is a figure showing shaper/meter block registers,. 


T=af 

5 


15 


Figure 494 is a figure showing read FID1 memory. 




16 


Figure 495 is a figure showing write FID1 memory when using shapen 




17 


Figure 496 is a figure showing read FID2 memory. 


S 


18 


Figure 497 is a figure showing write FID2 memory.. 


; e-.s 


19 


Figure 498 is a figure showing read shp slot memory^ 


3 ; 


20 


Figure 499 is a figure showing write shp_slot memory. 




21 


Figure 500 is a figure showing read rate id memory^ 




22 


Figure 501 is a figure showing write rate_id memory. 




23 


Figure 502 is a figure showing setup connection command,. 




24 


Figure 503 is a figure showing read FID1 memory. 




25 


Figure 504 is a figure showing write FID1 memory.. 




26 


Figure 505 is a figure showing read FID2 memory. 




27 


Figure 506 is a figure showing write FID2 memory. 




28 


Figure 507 is a figure showing read threshold (internal) memory. 




29 


Figure 508 is a figure showing write threshold (internal) memory^ 




30 


Figure 509 is a figure showing readmod write meter. 




31 


Figure 510 is a figure showing setup connection command. 
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1 Figure 51 1 is a figure showing sceduler block registers. 

2 Figure 512 is a figure showing read FID next memory^ 

3 Figure 513 is a figure showing write FID next memory. 

4 Figure 5 14 is a figure showing read qos parameters. 

5 Figure 5 1 5 is a figure showing write qos parameters. 

6 Figure 516 is a figure showing read qos descriptor. 

7 Figure 5 17 is a figure showing write qos descriptor 

8 Figure 5 1 8 is a figure showing read port parameters^ 

9 Figure 5 1 9 is a figure showing write port parameters^ 

10 Figure 520 is a figure showing read port descriptor 

1 1 Figure 521 is a figure showing write port descriptor 

1 2 Figure 522 is a figure showing read port next 
%g 13 Figure 523 is a figure showing write port next. 

14 Figure 524 is a figure showing init qos. 

1 5 Figure 525 is a figure showing init port 

5 16 Figure 526 is a figure showing parameters per flowid^ 

^ 1 7 Figure 527 is a figure showing mx2 external memory. 

3 

2 18 

g 19 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

H; 20 The present invention will now be described in detail with reference to a few 

i 21 preferred embodiments thereof as illustrated in the accompanying drawings. In the 

22 following description, numerous specific details are set forth in order to provide a 

23 thorough understanding of the present invention. It will be apparent, however, to one 

24 skilled in the art, that the present invention may be practiced without some or all of these 

25 specific details. In other instances, well known process steps and/or structures have not 

26 been described in detail in order to not unnecessarily obscure the present invention. 
27 

28 System Overview 

29 The Maximus implements a multi-service approach with dynamic bandwidth 

30 allocation among the various flows carrying the different types of traffic. The Maximus 

31 device is designed such that it can be used in different applications that fit the needs of 

21 
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various system architectures. Two of the more popular applications are traffic manager 
and SAR solutions for OC-192 line cards in routing and switching systems. In one 
embodiment, the Maximus device is implemented in the form of two integrated circuit 
chips, designated the MX-1 and MX-2. In another embodiment, the entirety of the 
Maximus device is implemented on one integrated circuit chip (the circuitry of the MX-1 
and MX-2 integrated circuits are integrated onto a single integrated circuit chip). 

Figure 1 depicts an exemplary system level overview, illustrating how the 
Maximus device may enable OC-192 line cards in a router or switch system to receive 
different types of traffic, to dynamically allocate bandwidth among the various flows, to 
adapt the incoming traffic to the requirements of the cell-based or packet-based switch 
fabric, and to convert among the different traffic types for output. The term "router" as it 
is used in this document includes, but is not limited to, packet routers (for example, IP 
packet routers) and cell switches (for example, what are commonly called ATM 
switches). 

In Figure 1, each line card can receive both ATM and packet traffic coming in on 
different flows across a fiber optic connection. The Maximus device on each line card, 
acting as an ingress device interfacing with a framer device, can receive, buffer and adapt 
the incoming traffic to the switching requirements of either the cell or packet switch 
fabric. On the egress side, the traffic can be sent out to the framer device either in the 
same traffic type (e.g., ATM in, ATM out) or in a different traffic type (e.g., ATM in, 
packet out or packet in, ATM out). With regard to dynamic bandwidth allocation, the 
Maximus offers extensive QoS capabilities to shape and schedule the flows as desired 
according to their QoS priorities. 

Figures 2 and 3 below are two exemplary linecard views, illustrating how the 
Maximus device may be employed on an OC-192 line card operating with either a cell- 
based switch fabric (Figure 2) or a packet-based switch fabric (Figure 3). In these 
Figures, the traffic management or SAR solution is accomplished using two separate 
chips. 

In Figure 2, a traffic management solution in conjunction with a cell-based switch 
fabric is illustrated. Four OC-48 optics couple with a quad OC-48 Framer to achieve an 
aggregate OC-192 data rate. This is not a requirement, however, as the Maximus Ingress 
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1 Traffic Manger and the Maximus Egress Traffic Manager can also directly couple with 

2 an OC-192 framer to receive data at 10 Gbps. The Maximus supports 64 incoming or 

3 outgoing ports and up to 1 million separate flows in each direction. The interface 

4 between the Framer and Maximus Traffic Manager chips is SPI-4 (System Packet 

5 Interface-4), phase 2 (i.e., LVDS running at 400 MHz). Data storage and buffering 

6 employs SDRAMs, while pointers and tables are stored in SSRAMs. The Maximus 

7 Traffic Manager chips interface with a cell-based switch fabric in the example of Figure 

8 2. 

9 In the example of Figure 3, the Maximus Ingress Traffic Manger and the 

10 Maximus Egress Traffic Manager are employed in the SAR configuration. For example, 

1 1 the Maximus may receive ATM traffic, reassembles the ATM cells into packets along the 

12 ingress path to switch through a packet-based switch fabric. On the egress path, the 
Q 13 packets may be segmented back into ATM cells for output to the quad OC-48 framers. 
J; 14 However, due to the multi-service capabilities offered, packets may also be received and 
^ 1 5 switched through the packet-based switch fabric. The data in these packets (whether 

y 16 received as ATM cells or packets) may be output to the Framer as either ATM cells or 

a f? 17 packets. 

L 18 

m 19 A pplications 

O 20 Figure 4 shows the various applications, along with their combinations of 

O 21 incoming and outgoing traffic types. 

^ 22 Application types 0-3 in Figure 4 represent the ingress traffic manager 

23 applications. Application types 8-1 1, on the other hand, represent the egress traffic 

24 manager applications. These ingress and egress traffic management application types (0- 

25 3 and 8-1 1) are graphically represented in Figure 5. Note that traffic management is 

26 offered not only for packets but also for ATM cells and AAL5 cells in a single pair of 

27 Maximus chips (ingress/egress pair). This reflects the multi-service approach of the 

28 Maximus. 

29 Figure 5 A shows uses of the Maximus integrated circuit chip with a cell-based 

30 switch fabric. The dashed box labeled "MX-1" indicates the contents of one integrated 

31 circuit. Certain other parts of the integrated circuit (for example, the shaper block, the 

23 
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scheduler block, and the memory management block) are omitted from Figure 5 A in 
order to clarify the illustration. These other parts are described in further detail below in 
the later part of this patent document. The darkening of a block in Figure 5 A indicates 
that the block is substantially disabled in that it is not performing its primary function. 
Blocks that are not darkened in Figure 5A are enabled to perform their primary functions. 

Application types 4-6 and 12-14 represent the SAR application types, along with 
their combinations of incoming and outgoing traffic types. These SAR application types 
(4-6 and 12-14) are graphically represented in Figure 6. Note that these SAR application 
types can handle not only segmentation of packets and reassembly of cells but can also 
support ATM cell encapsulation and packet bypass (application types 4, 6, 12, and 14). 
Again, this reflects the intelligent multi-service approach of the Maximus. 

Figure 6A shows uses of the Maximus integrated circuit chip with a packet-based 
switch fabric. The dashed box labeled "MX-1" indicates the contents of one integrated 
circuit. Certain other parts of the integrated circuit (for example, the shaper block, the 
scheduler block, and the memory management block) are omitted from Figure 6A in 
order to clarify the illustration. These other parts are described in further detail below in 
the later part of this patent document. The darkening of a block in Figure 6 A indicates 
that the block is substantially disabled in that it is not performing its primary function. 
Blocks that are not darkened in Figure 6A are enabled to perform their primary functions. 

The data formats for the various application types set forth in Figures 5, 5 A, 6 and 
6A are discussed below, with reference to the data format drawings of Figures 7-20. 

Data Formats 

Figure 7 illustrates the data format for Ingress Application Type 0 (ATM => 
ATM). In this application, the incoming data via the incoming SPI interface is ATM 
cells (52 bytes, which includes 4 bytes of ATM header and 48 bytes of data). Four bytes 
of pad is added prior to sending the incoming data to the lookup block (since the 
Maximus internal data path is 64-bit or 8-byte wide). The segmentation block adds 8 
bytes of pad to allow the memory manager to pass the 64-byte memory cells to SDRAMs 
for storage. The reassembly block adds either 8 or 16 bytes of header to the 64-byte 
memory cell prior to sending to the cell-based switch (via the outgoing SPI block). The 
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reader is invited to review egress Application type 8 (Figure 14) for the analogous 
process on the egress side, if the data is to be output to the Framer by the egress Maximus 
chip as ATM cells. Egress application type 8 is discussed later herein. 

Figure 8 illustrates the data format for Ingress Application Type 1 (ATM => 
MPLS packet). In this application, the incoming data via the incoming SPI interface is 
ATM cells (52 bytes, which includes 4 bytes of ATM header and 48 bytes of data). Four 
bytes of pad is added prior to sending the incoming data to the lookup block (since the 
Maximus internal data path is 64-bit or 8-byte wide). The segmentation block removes 
the 4-byte ATM header and adds 12 more bytes of pad to allow the memory manager to 
pass the 64-byte memory cells to SDRAM for storage. The reassembly block adds either 
8 or 16 bytes of header to the 64-byte memory cell prior to sending to the cell-based 
switch (via the outgoing SPI block). The reader is invited to review egress Application 
type 9 (Figure 15) for the analogous process on the egress side, if the data is to be output 
to the Framer by the egress Maximus chip as MPLS packets. Egress application type 9 is 
discussed later herein. 

Figure 9 illustrates the data format for Ingress Application Type 2 (MPLS packet 
=>ATM). In this application, the incoming data via the incoming SPI interface is MPLS 
packet (Nxl6 bytes burst). The packet data is passed onto the lookup engine. The 
segmentation block segments the packet data into 48-byte AAL5 cells and appends 16 
bytes of pad to allow the memory manager to pass the 64-byte memory cells to SDRAM 
for storage. The reassembly block adds either 8 or 16 bytes of switch header to the 64- 
byte memory cell prior to sending to the cell-based switch (via the outgoing SPI block). 
The reader is invited to review Application type 10 (Figure 16) for the analogous process 
on the egress side, if the data is to be output to the Framer by the egress Maximus chip as 
ATM cells. Egress application type 10 is discussed later herein. 

Figure 10 illustrates the data format for Ingress Application Type 3 (Packet => 
Packet). In this application, the incoming data via the incoming SPI interface is packet 
(Nxl6 bytes burst). The packet data is passed onto the lookup engine. The segmentation 
block segments the packet data into AAL5-like 64-byte cells. In this segmentation, CRC- 
32 is generated and an AAL5-like trailer is added to the end of the last 64-byte cell. The 
entire 64-byte AAL5-like cell is passed onto the memory manager for storage in 
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1 SDRAM. The reassembly block adds either 8 or 16 bytes of switch header to the 64-byte 

2 memory cell prior to sending to the cell-based switch (via the outgoing SPI block). The 

3 reader is invited to review Application type 1 1 (Figure 17) for the analogous process on 

4 the egress side, if the data is to be output to the Framer by the egress Maximus chip as 

5 packets. Egress application type 1 1 is discussed later herein. 

6 Figure 1 1 illustrates the data format for Ingress Application Type 4 (ATM 

7 Encapsulation). In this application, the incoming data via the incoming SPI interface is 

8 ATM cells (52 bytes, which includes 4 bytes of ATM header and 48 bytes of data). Four 

9 bytes of pad is added prior to sending the incoming data to the lookup block (since the 

10 Maximus internal data path is 64-bit or 8-byte wide). The segmentation block adds 8 

1 1 bytes of pad to allow the memory manager to pass the 64-byte cell to SDRAM for 

12 storage. The reassembly block removes 8 bytes of pad, adds either 8 or 16 bytes of 
□ 1 3 switch header prior to sending the resultant 64-byte packet to the packet-based switch 

14 (via the outgoing SPI block). The reader is invited to review egress Application type 12 

^ 15 (Figure 1 8) for the analogous process on the egress side, if the data is to be output to the 

l2 16 Framer by the egress Maximus chip as ATM cells. Egress application type 12 is 

rf 17 discussed later herein. 

s 1 8 Figure 12 illustrates the data format for Ingress Application Type 5 (Reassembly). 

m 19 In this application, the incoming data via the incoming SPI interface is ATM cells (52 

H 20 bytes, which includes 4 bytes of ATM header and 48 bytes of data). Four bytes of pad is 

O 21 added prior to sending the incoming data to the lookup block (since the Maximus internal 

^ 22 data path is 64-bit or 8-byte wide). The lookup block removes the 4-byte ATM header 

23 and passes the 48-byte ATM data to the segmentation block, wherein 16 bytes of pad are 

24 added to allow the memory manager to pass the 64-byte cells to SDRAM for storage. 

25 The reassembly block maps the 48-byte ATM cells stored in memory as 64-byte cell into 

26 packets by stripping 16 bytes of pad from each memory cell and adding a switch packet 

27 header before sending the resultant packet to the packet-based switch via the outgoing 

28 SPI. The reader is invited to review egress Application type 13 (Figure 19) for the 

29 analogous process on the egress side, if the data is to be output to the Framer by the 

30 egress Maximus chip as ATM cells. Egress application type 13 is discussed later herein. 
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1 Figure 1 3 illustrates the data format for Ingress Application Type 6 (Ingress 

2 Packet Bypass). In this application, the incoming data via the incoming SPI interface is a 

3 packet burst (Nxl 6 bytes). The packet data is passed to the lookup block, which 

4 subsequently passes the packet data to the segmentation block. The segmentation block 

5 segments the packet data into AAL5-like 64-byte cells. In this segmentation, CRC-32 is 

6 generated and an AAL5-like trailer is added to the end of the last 64-byte cell. The entire 

7 64-byte AAL5-like cell is passed onto the memory manager for storage in SDRAM. The 

8 reassembly block maps the 64-byte AAL5-like memory cell into packets by removing the 

9 AAL5 encapsulation and may add 8 or 1 6 bytes of switch packet header. The 

1 0 reassembled packet, including the 8/16 bytes of switch packet header is passed on to the 

1 1 packet-based switch via the outgoing SPI. The reader is invited to review egress 

12 Application type 14 (Figure 20) for the analogous process on the egress side, if the data is 
O 13 to be output to the Framer by the egress Maximus chip as packets. Egress application 

%j 14 type 14 is discussed later herein. 

^ 1 5 Figure 1 4 illustrates the data format for Egress Application Type 8 (ATM 

yj i g =>ATM). In this application, the incoming data (from the switch) via the incoming SPI 

E 1 7 interface is a switch cell which comprises 8/1 6 bytes of switch header, 4 bytes of ATM 

L 18 header, 48 bytes of ATM data, and 12 bytes of pad. The switch header is removed by 

3 1 9 the lookup block, resulting in a 64-byte cell. The 64-byte cell is then passed through the 

2 20 segmentation block to the memory manager to be stored in SDRAM as a 64-byte memory 

O 2 1 cell. The reassembly block translates the ATM header, removes 8 bytes of pad (leaving 4 

~ 22 bytes of pad behind to facilitate sending the resultant 52-byte ATM cell + 4 bytes of pad 

23 to the outgoing SPI interface (since the Maximus internal data path is 64-bit or 8-byte 

24 wide). These 4 bytes of pad are later removed to allow the 52-byte ATM cell to be sent 

25 out to the Framer (via the outgoing SPI block). 

26 Figure 1 5 illustrates the data format for Egress Application Type 9 (ATM 

27 =>MPLS packet). In this application, the incoming data (from the switch) via the 

28 incoming SPI interface is a switch cell which comprises 8/16 bytes of switch header, 48 

29 bytes of AAL5 data, and 1 6 bytes of pad. The switch header is removed by the lookup 

30 block, resulting in a 64-byte cell. The 64-byte cell is then passed through the 

3 1 segmentation block to the memory manager to be stored in SDRAM as a 64-byte memory 
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1 cell. The reassembly block strips the 1 6 bytes of pad, performs a MPLS operation of 

2 adding the MPLS packet header based on the contents of the FlowID lookup table before 

3 sending the resultant packet out to the Framer (via the outgoing SPI block). 

4 Figure 16 illustrates the data format for the Egress Application Type 10 (MPLS 

5 packets =>ATM). In this application, the incoming data (from the switch) via the 

6 incoming SPI interface is a switch cell which comprises 8/16 bytes of switch header, 48 

7 bytes of AAL5 data, and 16 bytes of pad. The switch header is removed by the lookup 

8 block, resulting in a 64-byte cell. The 64-byte cell is then passed through the 

9 segmentation block to the memory manager to be stored in SDRAM as a 64-byte memory 

10 cell. The reassembly block adds the 4-byte ATM header and appends 4-byte of pad, as 

1 1 well as strip the 16-byte of pad. These 4 newly added pad bytes are subsequently 

1 2 removed to allow the 52-byte ATM cell to be sent to the Framer (via the outgoing SPI 
O 13 block). 

S| 1 4 Figure 1 7 illustrates the data format for the Egress Application Type 1 1 (packet 

.« 1 5 => packet). In this application, the incoming data (from the switch) via the incoming 

y 1 6 SPI interface is a switch cell comprising 64 bytes of AAL5-like data and an 8/1 6-byte 

2 1 7 switch header. The switch header is removed by the lookup block, resulting in a 64-byte 

L 1 8 cell. The 64-byte AAL5-like cell is then passed through the segmentation block to the 

5 1 9 memory manager to be stored in SDRAM as a 64-byte memory cell. The reassembly 

J 20 block maps the 64-byte AAL5-like cells into a packet, and adds a 8/1 6-byte packet 

O 2 1 header. The reassembly block also checks the CRC in the AAL5 trailer before removing 

^ 22 the AAL5-like encapsulation. The resultant packet is subsequently passed on to the 

23 Framer (via the outgoing SPI block). 

24 Figure 1 8 illustrates the data format for the Egress Application Type 12 (ATM de- 

25 encapsulation). In this application, the incoming data via the incoming SPI interface is a 

26 switch packet, which comprises 8/16 bytes of switch header, 4 bytes of ATM header, 48 

27 bytes of ATM data, and 4 bytes of pad. The switch header is removed by the lookup 

28 block, resulting in 56-byte cell. The 56-byte cell is passed to the segmentation block, 

29 which adds 8 bytes of pad to allow the memory manager to pass the 64-byte cell to 

30 SDRAM for storage. The reassembly block translates the ATM header, strips out 8 bytes 

3 1 of pad (leaving 4 bytes to allow passing through the 8-byte/64-bit data path from the 
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1 reassembly block to the outgoing SPI block). These last four bytes of pad will be 

2 removed, allowing the outgoing SPI to send the 52-byte ATM cell to the Framer. 

3 Figure 19 illustrates the data format for the Egress Application Type 13 

4 (Segmentation). In this application, the incoming data via the incoming SPI interface is a 

5 packet burst, including the 8/1 6 bytes of switch header. The packet data is passed onto 

6 the segmentation block via the lookup block. At the segmentation block, the packet data 

7 is segmented into 48-byte AAL5 cells, and 16 bytes of pad are added to allow the 

8 memory manager to pass the 64-byte cells to SDRAM for storage. The reassembly block 

9 removes the 16 bytes of pad, adds the 4-byte ATM header to the 48-byte ATM data, and 

10 pads with 4 additional bytes to allow passing through the 8-byte/64-bit data path from the 

1 1 reassembly block to the outgoing SPI block. These last four bytes of pad will be 

p 12 removed subsequently, allowing the outgoing SPI to send the 52-byte AAL5 cell to the 

J 1 *; 13 Framer. 

H 1 14 Figure 20 illustrates the data format for the Egress Application Type 14 (Egress 

M 15 Packet Bypass). In this application, the incoming data via the incoming SPI interface is a 

P 16 packet burst, which comprises 8/16 bytes of switch header and 64 bytes of AAL5 data. 

5 1 7 The packet data is passed onto the segmentation block via the lookup block. At the 

« 1 8 segmentation block, the packet data is segmented into 64-byte AAL5-like cells for 

O 1 9 storage in SDRAM memory. The reassembly block maps the 64-byte AAL5-like cells 

□ 20 into packet, and adds a packet header of 0-16 bytes. It also checks the CRC in the AAL5 

^ 2 1 trailer before removing the AAL5 encapsulation. The packet is then passed to the Framer 

22 via the outgoing SPI block. 

23 

24 Block Diagram 

25 Figure 21 is a high-level block diagram of the Maximus 1 integrated circuit chip, 

26 the Maximus 2 integrated circuit chip, and various memory devices coupled to the 

27 Maximus 1 and 2 integrated circuit chips. All of these integrated circuits are present, in 

28 one embodiment, on a line card of a router. The functionality of the Maximus 1 (MX-1) 

29 and Maximus 2 (MX-2) integrated circuit chips may be disposed on two different 

30 integrated circuit chips as illustrated in Figure 21 . Alternatively, the circuitry of the 

29 
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Maximus 1 and Maximus 2 integrated circuit chips is combined such that all that circuitry 
is disposed on a single integrated circuit chip. 

Features 
General Features 

• Supports full duplex lOGbps performance 
□ Protocols and internetworking 
o ATM 



■ Operation and Management (OAM) cells 

■ Signaling support 

■ VP/VC translation 

■ ATM to packet internetworking with Segmentation and reassembly 

■ VC merging support 
o Packets 

■ MPLS 



• MPLS tag switching 

• IP to ATM internetworking with AAL5 segmentation and 
re-assembly. 

• MPLS header generation. Used for MPLS tunneling and 
aggregation 

• MPLS merging several flows from different ports merged 
to single output. 



■ Any packet with MPLS-like header 
o Management support 

■ SNMP, MIB counters 

■ Alarms and Events 

o Multicast support for both ATM and Packets 
• Applications supported: 

o Ingress Traffic Manager 

■ ATM Cells in, ATM Cells out. 
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1 ■ ATM (AAL5) Cells in, packets out. 

2 ■ MPLS Packets in, ATM (AAL5) Cells out. 

3 ■ MPLS Packets in, MPLS Packets out. 

4 o Egress Traffic Manager 

5 ■ ATM Cells in, ATM Cells out. 

6 ■ ATM (AAL5) Cells in, Packets out. 

7 ■ MPLS Packets in, ATM (AAL5) Cells out. 

8 ■ MPLS Packets in, MPLS Packets out. 

9 o Reassembler 

1 o ■ ATM (AAL5) cells in, packets out 

I \ m ATM cells in, ATM cell encapsulated inside packet out. 

12 ■ MPLS Packets in, MPLS Packets out. 

13 o Segmentation 

1 4 ■ MPLS Packets in, ATM (AAL5) Cells out. 

1 5 ■ ATM cell encapsulated inside packet in, de-encapsulated ATM cell 

16 out. 

1 7 ■ MPLS Packets in, MPLS Packets out. 

18 • Interfaces supported 

19 o Generic CPU Interface 

20 o SPI-4 phase-2 interfaces for both packets and cells. 

21 ■ 16-bit, 400 MHz, DDR interface. 

22 ■ Supports up to 64 ports (any combination of STS- 1 - STS- 1 92). 

23 ■ SPI-4 used for both incoming and outgoing interfaces. 

24 • Traffic management and QoS mechanisms 

25 o Traffic Management on Ingress 

26 ■ Priority scheduling with weighted queuing 

27 • Per Flow per egress port per class scheduling (up to lk 

28 queues) 

29 • Flow control with the switch 

30 o Traffic Management on Egress 

3 1 ■ Traffic shaping through dual leaky bucket 
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■ Traffic scheduling 

• 64 Utopia level 4 ports 

• 8 Class of service 

• GCRA scheduling 

■ Weighted fair queuing 
o VP tunneling support 

o CBR tunneling 
o MPLS tunneling 
CPU Interface 

• Supports CPU interface for management and control information. 

• Supports 32-bit CPU bus interface. 

• Support Mux/Non-Mux and Big Endian/Little Endian modes 

• Supports interrupt generation. 

o Hierarchical interrupt structure 
Incoming SPI-4 Interface 

• Supports the SPI-4 phase II specifications for 10 Gbps rate. 

• Unpacks incoming 16-bit DDR data at 400 MHz into 64-bit wide word at 200 
MHz. 

• Performs diagonal interleaved parity checking. 

• Extracts in-band port address, SOP, EOP and error-control code. 

• In-band start-of-FIFO status signal. 

• 200 MHz for the FIFO status transfer. 

• Supports maximum 64 ports. 

• Generates 2-bit FIFO status back to the transmit device using programmed port 
calendar. 

o Programmable port calendar with STS-1 granularity. 
Lookup Engine 

• Supports up to 1M flows. 

• Supports up to 64 input ports. 
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1 • Tables and configuration registers loaded via the processor interface. 

2 • Lookup Algorithm 

3 o Label generator selects any 6 bytes from the first 32 bytes of the packet. 

4 o Label generator is programmable on a per port basis. 

5 o The label generator has one byte of granularity. 

6 o Hash generator creates 2 hash values or slots in parallel from a key using 2 

7 separate CRC polynomials as hash functions. 

8 o Four bins per bucket to minimize collisions. 

9 o Hash table lookup constrained to resolve within 4 tries. 

I o o Overflow CAM of 256 or 64 entries, the largest available, for lookups that 

I I fail to resolve in 4 tries. 

12 o Fully pipelined, one lookup every 8 cycles, 

y 1 3 • Supports the following flows with the following headers: 

H 14 o MPLS with PPP encapsulation. 

.2 1 5 o MPLS with LLC/SNAP encapsulation. 

| 16 o MPLS with Frame Relay encapsulation.. 

u 17 o Custom switch headers for traffic manager application. 

^18 o Supports bypass lookup using MPLS-like header. 
J 19 Segmentation Engine 

M= 20 • Segments incoming packet data into cells on a per port basis 

fj 21 o Supports a maximum of 64 external ports 

22 o Supports an additional port for data from processor 

23 o Segments incoming packet data into ATM cells 

24 o Appends AAL-5 trailer for ATM segmentation 

25 o Segments incoming packet data into Switch cells of 64 bytes 

26 o Appends AAL-5 like trailer for non-ATM segmentation 

27 • Supports programmable maximum packet length of 1 6K bytes. 

28 • Gathers per port statistics 

29 o 32-bit counter for incoming packets. 

30 o 48-bit counter for byte count. 

31 o 1 6-bit counter to count discarded packets. 
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1 • Tags bad packets from SPI, to be discarded by per flow queue. 

2 • Internal SRAM to store per port segmentation context and data 

3 Memory Manager 

4 • Uses two channels of 1 28-bit (64-bit DDR) SDRAM 

5 • Manages read and write requests to all eight banks separately to maximize 

6 utilization. 

7 • Uses 1 28kb of internal SRAM for temporary storage of data to and from 

8 SDRAM. 

9 • Sopisticated load-sharing algorithm to distribute traffic over various banks. 

10 Reassembly and Header Processing Engine 

1 1 • Types of operation: 

12 o ATM cells to ATM cells no action required with translation 

jests. 

*i 13 o ATM cells to ATM cells with translation 

*y 

^14 o ATM cells to Switch cells 

N! 

^ 15 o Switch cells to Switch cells 

J 1 6 o Switch cells to Packet Data 

N= 17 o ATM cells to Packet Data 

g 18 o AAL-5 cells to ATM cells, appending ATM header 

J 1 9 o AAL-5 cells to Packet Data 

U 20 Header Processing Features 

^ 2 1 • Supports a maximum of 1M total flows. 

22 • Types of lookup supported 

23 o No lookup, pass-through traffic 

24 o ATM processing 

25 ■ ATM Header append 

26 ■ VPI, VCI header translation 

27 ■ Encapsulate raw cells and OAM cells into packets for routing 

28 through packet fabrics. 

29 ■ Encapsulate ATM cells into packets for routing through packet 

30 fabric. 

31 o MPLS processing 
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■ MPLS push/pop 

■ MPLS header translation 
Reassembly Features 

• Reassembles incoming cells into packets on a per port basis 

o Supports a maximum of 64 external ports 
• o Supports an additional port for data to processor 
o Reassembles incoming ATM cells into packet data, 
o Strips AAL-5 trailer for ATM reassembly 
o Segments incoming Switch cells of 64 bytes into packet data, 
o Strips AAL-5 like trailer for non-ATM reassembly. 

• Supports programmable maximum packet length of 1 6K bytes. 

• Gathers per port statistics 

o 32-bit counter for outgoing packets. 

o 48-bit counter for byte count. 

o 16-bit counter to count discarded packets. 

• Tags bad packets to SPI, to be discarded by downstream device. 

• Internal SRAM to store per port segmentation context and data 
Outgoing SPI-4 Interface 

• Supports SPI-4 phase II spec at 10 Gbps rate. 

• Packs the 64-bit data at 200 MHz to 16 bit DDR data at 400 MHz. 

• Multiplexes the data and the control onto the SPI-4 bus. 

• Diagonal Interleaved parity generation. 

• Programmable data transfer burst sizes of multiple 1 6 bytes. 

• Passes the FIFO status information to the scheduler to stop transferring. 

• Schedules training sequence for data path de-skew. 

• Supports maximum 64 ports 

• Receives 2-bit FIFO status from the receive device using programmed port 
calendar. 



o Programmable port calendar with STS-1 granularity. 
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1 Per Flow Queue Engine 

2 • See below, page 88. 

3 Traffic Shaper and Traffic Meter 

4 • See below, page 129. 

5 Output Scheduler 

6 • See below, page 118. 
7 

8 Signals 

9 See Figure 22. 



10 

11 MX 1 CPU Interface 

12 This block is responsible for three different functions: 



13 • Connects between the blocks and the CPU via the CPU interface. It decodes the 

H 14 address to Figure which block to access and generate the chip select to each 

i% 15 block. 

* 16 • Generates global synchronization signals to the rest of the blocks. 

2 17 • Holds general control registers like version register, mode register etc. 

m 18 

^ 19 Features 

Q 20 1 . Address bus of 1 0 bits per chip. 

^ 21 Bits 0-5 are used as an address inside the block (up to 64 registers per block) 

22 Bits 9-6 are used to select a block (up to 16 blocks per chip) 

23 2. Data bus of 32 bits per chip. 

24 Externally this bus is b-directional 

25 Internally this bus is split to 2 busses: DATA IN and DATA_OUT. 

26 3. Chip select signal is the qualifier for data, address and read/write busses. 

27 4. The CPU interface can support multiplexed CPU interface or non-multiplexed CPU 

28 interface. When using multiplexed interface, the address and data are folded to the 

29 DATA[31:0]bus. 

30 5. CPU accesses all registers in all blocks via direct read/write commands. 
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1 6. CPU accesses all internal/external memories in all blocks via indirect accesses using 

2 the general-purpose registers dedicated for these accesses. 

3 7. Blocks addresses can be found in Figure 23. 

4 8. Generates a global sync signal once every 520 cycles of the 200MHz clock (65 ports, 

5 8 cycles per port). 
6 

7 Functional Description 

8 The CPU IF block is used to interface with the CPU. The CPU IF block decodes 

9 the incoming address bus to generate a unique chip-select signal to each block. The CPU 

10 IF block also holds general registers that contain general parameters and modes for the 

1 1 entire chip. Basically any logic that is not related directly to a specific block is in this 

12 block. 

5 1 3 The CPU IF block can operate in two modes depending on an input pins to select 

H 14 between them: 

Ji 15 • Regular CPU access: separate busses for data and for address. In this case address 

j** 16 bus is qualified be the chip-select signal 

M= 1 7 • Multiplexed CPU access: one bus is multiplexed for address and for data. In this 

q 18 case address is qualified by the ALE signal. 

2 1 9 The CPU interface is based on Intel's microprocessor interface (when multiplexing 

C 20 address/data bus). The following timing diagrams defines the characteristics of the CPU 

H 2 1 interface for Read and for Write accesses: 
22 

23 Read access characteristics 

24 See Figures 24 and 25. 

25 • RDWRL signal has the same timings as the ADDR bus. 

26 • A valid read cycle is defined when RDWR L is latched as high on the negative edge 

27 oftheCS_L signal. 

28 • In non-multiplexed address/data bus architecture, ALE should be held high so 

29 parameters t Sa ir, t Ha ir, Wl, tsir and t H i r are not applicable. 

30 • In our implementation the address is latched on the negative edge of the CS_L 

3 1 therefore W is not applicable. 
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1 

2 Write access characteristics 

3 See Figures 26 and 27. 

4 • RDWR_L signal has the same timings as the ADDR bus. 

5 • A valid write cycle is defined when RDWR L is latched as low on the negative edge 

6 oftheCS_L signal. 

7 • In non-multiplexed address/data bus architecture, ALE should be held high so 

8 parameters t Sa iw, t Ha iw 3 Wl, tsiw and t H iw are not applicable. 

9 • In our implementation the address is latched on the negative edge of the CS_L 
10 therefore t Haw is not applicable. 

11 

^ 12 Timing diagram of signals to blocks 

M 13 The diagram in Figure 28 describes the timings of the CPU interface to each 

14 block. Note that chip-select signal is now asserted for only one cycle of the 200MHz 

* 15 clock. For a write or read cycle, all parameters are valid and stable before chip-select is 

1 6 asserted. When a read cycle occurs data is driven from the block after a maximum of 3 

f 17 clock cycles of 200MHz clock. 

U 18 

m 

O 19 GSYNC - Synchronization signal 

« 20 GSYNC signal is a one-cycle pulse in the 200MHz clock, which is asserted once 

M= 21 every 520 cycles. The SYNC signal is used for synchronization between the blocks and 

22 between the chips as well as to generate the slot-count and the CPU-port indication. See 

23 Figure 29. 

24 Slot count and CPU port indication 

25 The signals slot_count and CPU_port are generated inside each block base on the 

26 GSYNC signal. Each block first needs to sample GSYNC and use the sampled version to 

27 generate the signals. See Figure 30. 
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2 Interface 

3 The CPU IF block generates chip-select signal per block, the rdwrj, addr and 

4 data busses are propagated to the blocks directly. The blocks perform the read or write 

5 command based on the rdwrj signal only when the chip-select signal per the block is 

6 asserted. The chip-select signal per each block is synchronous to the 200MHz clock 

7 domain already. Each block first samples the incoming signals and then uses them. This 

8 is being done to solve any timing issues that may occur. If during placement and routing 

9 the signals from the CPU IF to the blocks will violate the 5nSec budget, another set of 

10 FFs can be placed in the middle of the path. 

1 1 For a read cycle the CPU EF block selects the data busses coming in from the blocks 

1 2 based on the chip-select per block (the address). See Figure 3 1 . 

1 3 The address, data and rd/wr signals takes about 4 cycles of 200MHz to propagate 

14 from the chip's pins to the block (6 cycles are used for delay calculation). 2 cycles are 

1 5 needed to perform read or write accesses in the block. 

16 Therefore a total of 8 cycles are needed to perform a read or a write access. 
H 17 

5 

r% 18 Block Descriptions 

C 20 CPU Interface 

rf 2 1 The negative edge of the CS_L signal qualifies the ADDR[9:0] and the RDWRJL 

22 signals. The positive edge of the CS_L signal qualifies the DATA bus for read and for 

23 write accesses. The CS_L is basically an enable signal to latches. The circuit shown in 

24 Figure 32 shows the basic latching circuit and the synchronization to 200MHz clock 

25 domain: 
26 

27 Test Mux 

28 The Test Mux block is used to multiplex test mux outputs from all the blocks 

29 inside the chip. The output bus of the test mux go out to the pins that will be used for 

30 testing and debugging purposes. Refer to the Figure below for the Test Mux block 

3 1 diagram. The Tstmux_Blk_Sel register (refer to the Register section) is used to select 



1*1 
War 
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1 which block signals to be output to the test pins (Tstmux_pin[3 1 :0]). Each block inside 

2 the chip also has selection register (e.g. Seg_Tstmux_Sel, Ras_Tstmux_Sel) to select a 

3 group of internal signals to be observed. 

4 The test mux outputs of all the blocks should be clocked out of flip-flops (refer to 

5 the diagram below). The CPU_clk that is running at 200 Mhz, is used to clock the output 

6 of the test mux inside the CPU Interface block. It is also going out to a test pin called 

7 Tstmux_clk. See Figure 33. 



8 

9 Soft Reset 

10 Each block inside MX-1 chip can be reset individually by writing a ' 1' to its 

1 1 Soft_Reset register (refer to the Register section). Writing a '0' to the register will release 

12 the block from being reset and get it back to its normal operation. Each soft reset signal 
O 13 of each block is also coupled with the global power-on reset pin (POR L). Refer to 

SI 1 4 Figure 34 below. 
~ 15 

W 16 Input/Output Disable 

2 17 Each block in the MX-1 chip has 2 inputs coming from the CPU Interface block 

1 8 that are used to disable the input and output of the block. The Input_Disable and 

P 19 OutputJDisable registers (refer to the Register section) can be programmed to disable the 

U 20 input or/and output of a particular block. Refer to Figure 35 below. 
P 21 

pi 

22 Registers 

23 The following table describes the registers implemented in the CPU IF block. 

24 These registers act as control to the rest of the chip. The registers are implemented in the 

25 200MHz clock domain even if there is no need to do so since their value is fixed and 

26 stable during normal operation. See Figure 36. 
27 

28 Incoming SPI-4 

29 The Incoming System Packet Interface Level 4 (SPI-4) block receives 16-bit 

30 DDR ATM or Packet over SONET/SDH (POS) data from the Transmit Device (PHY or 

3 1 Switch based on ingress or egress application) and sends out-of-band FIFO status 
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1 information to the Transmit Device. The incoming SPI-4 interface block outputs data 

2 with control information to the lookup engine, and receives FIFO status from 

3 segmentation engine. Furthermore, the CPU can program the registers in the incoming 

4 SPI-4 interface block via CPU interface. 
5 

6 Features 



7 - Supports the SPI-4 phases II spec at 10 Gbps rate 

8 - Packs incoming 1 6-bit DDR data at 400MHz into 64-bit wide word at 

9 200MHz 

10 - Free buffer list to manage per port FIFO and absorb data while look up engine 

11 is doing the look up 

12 - Diagonal interleaved parity checking for data 

5 13 - Extracts in-band port address, SOP, EOP and error-control code and presents 

SJ 14 to the lookup engine 

~ 1 5 - Passes 2-bit FIFO status back to the transmit device in a round robin fashion 

^ 1 6 based on the port calendar 

U 17 - Programmable port calendar to output FIFO status 

L 18 - In-band start-of-FIFO status signal 

I 19 - 400 MHz for the FIFO status transfer 

O 

20 - De-skews +/- 1 bit time after running the training patterns 

rf 21 - Supports maximum 64 ports 

22 

23 Functional Description 

24 The incoming SPI-4 interface block receives data with in-band port address, 

25 delineation information and error control code from an external transmit device. It 

26 separates the data and control and passes them to the lookup engine. It also sends out-of- 

27 band flow control information to the transmit device when it receives the information 

28 from the segmentation block. See Figures 37, 38 and 39. 
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1 Block Description 

2 Bit Align 

3 The bit-align block adds programmable delays to both data and control lines to 

4 correct for relative skew differences of up to +/- 1 bit time. The programmable delays are 

5 controlled by the bit align control block which receives pre-defined training patterns to 

6 determine the adjustment. It outputs 32 bit SDR data with two control signals. The 

7 control signals indicate which 16-bit word is data or control word. See Figure 40. 
8 

9 Bit Align Control 

1 0 This block compares the received training patterns with the pre-defined patterns 

1 1 and determines the receive timing of the lines. Based on the timing, it generates the 

1 2 control signal the select different delays to de-skew the data and control lines. 

1 13 

Nj 14 Drop IDLE 

,J 1 5 This block drops the extra idle control words from the incoming data and writes 

SJ 1 6 only data, training pattern, and the first detected idle control word into the input FIFO. 

M= 1 7 This block has a state machine which powers up in the idle state and remains in that state 

□ 1 8 until data or the first idle control word is detected. For the first detected idle control word, 

2 19 the state machine advances to the state named FIRST IDLE; and for non-idle control 
p 20 word it advances to the state named SCAN. If the state machine is in the FIRST IDLE 
H 21 state, it advances to the SCAN state if non-idle control word is detected, otherwise, it 

22 goes back to the IDLE state and asserts first idle signal. If the state machine is in the 

23 SCAN state, it stays in this state until idle control word is detected and it then advances to 

24 the FIRST IDLE state. The incoming data and control words are allowed to pass through 

25 and written together with the control signals into the input FIFO when the state machine 

26 is in the SCAN or FIRST IDLE states. See Figure 41. 
27 

28 Input FIFO 

29 This input FIFO is a 256x34 dual-port SRAM running at 400MHz to absorb 

30 incoming SPI-4 data, control words, and control signals. The write operation is controlled 
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1 by the drop IDLE state machine, and the read operation is controlled by the receive state 

2 machine (RX SM) when the FIFO is not empty. 
3 

4 Rx State Machine 

5 The Rx state machine receives 32-bit words from the input FIFO with indications 

6 of data or control words of the two 1 6-bit words. It extracts the in-band control 

7 information, packs the data into 64-bit wide words, passes the training patterns to the bit 

8 align control block. It also sends the data and the control words to the parity-checking 

9 block to check the parity. 

1 0 A common control word format is used in both the incoming and outgoing 

1 1 interfaces. The Figure below describes the fields of the control word. When inserted in 

12 the data path, the control word is aligned such that its MSB is sent on the MSB of the 
5 13 incoming or outgoing data lines. A (payload) control word that separate two adjacent 
H 14 burst transfers may contain status information pertaining to the previous transfer, the 

J 1 5 following transfer, or both transfers. The table below shows a list of control word types. 

^ 16 Maximus would ignore reserved control when detected from the incoming SPI-4 

M- 17 interface. 

L 18 Figures 42 shows the SPI-4 control word format and Figure 43 and 44 contain the 

2 19 detailed description for the SPI-4 control word format. 

2 20 The Rx state transition table is shown in Figure 45 and is based on the types of 

rf 21 control words received. Since the state machine has to decode two 16-bit words in one 

22 cycle, the table lists all the possible 16-bit words combinations. The 2-bit Ctrl signal is 

23 derived from the input DDR signal RCTL. Bit 1 is for the word 1 and bit 0 for word 0. 1 

24 indicates a control word and 0 is a data word. 

25 I ctrl = Idle control word detected P_ctrl = Payload control word detected 

26 T_ctrl = Training control word detected 
27 

28 Parity Checking 

29 The parity-checking block calculates 4-bit Diagonal Interleaved Parity (DIP-4) 

30 and compares the calculated DIP-4 with the one received in the control word. Then it 
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1 reports the error. The extent of the parity calculation is shown in Figure 46. A functional 

2 description of calculating the DIP-4 code is given in Figure 47. 
3 

4 Sync FIFO 

5 This FIFO is an 8x64 dual-port SRAM used to synchronize the 200MHz 

6 data/controls from the incoming SPI-4 to the internal clock. The write operation is 

7 controlled by the RX_SM and the read operation is controlled by the per_port control 

8 block. The per_port control reads from the sync FIFO as long as it is not empty. 
9 

10 SPI TBL 

1 1 The SPI TBL is a table of 64 entries, one for each port. The table contains control 
m 12 information for the incoming data burst for each port. 

1 13 

4 14 FBL 

j 1 5 The FBL (Free Buffer List) block is an 84x1 0 single-port SRAM containing the 

1 6 pointers to the data FIFO that are free to accept data. 

- 17 

2 18 Data FIFO 

19 The data FIFO is a 672x64 dual-port SRAM consists of 84 blocks of 64 bytes. It 

20 is used to receive data for each incoming port and only send the data out if it has 

21 accumulated 64 bytes or reached end of packet. 
22 

23 Calendar 

24 The calendar block contains a port calendar conFigured at the startup time. An 

25 example of a port calendar is shown in Figure 48. 

26 The calendar block receives per-port-basis FIFO status from the segmentation and 

27 encodes the status of each port in a 2-bit data structure. Then it sends the status back to 

28 the transmit device in a round robin fashion based on the port calendar. The "11" pattern 

29 is reserved for in-band framing. A DDP-2 odd parity is sent at the end of each complete 

30 sequence, immediately before the "11" framing pattern. See Figure 49. A description of 

31 calculating DIP-2 is shown in Figure 50. 
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1 

2 Memory Description 

3 

4 Summary of Internal Memories: 

5 See Figure 51. 

6 Interface 

7 Interface Ports 

8 See Figure 52. 
9 

10 Register Description 

11 See Figure 53. 
12 

13 Principles of Operation 

14 The incoming SPI block receives 16-bit DDR data, 1-bit DDR control and a 400 

1 5 MHz clock from the LVDS pins of the Maximus chip. It first converts the 1 6-bit DDR 

1 6 data words into 32-bit SDR data words and a 1 -bit DDR control into a 2-bit SDR control. 

1 7 The 2-bit control indicates which 1 6-bit word within a 32-bit word is the control word or 

1 8 data word. Based on the 2-bit control, the Rx state machine starts to extract the in-band 

19 port address, start/end-of-packet indication and error-control code from the 32-bit data 

20 path and packs the 1 6-bit data words into 64-bit data words. 

21 Subsequently, the incoming SPI block outputs the 64-bit data words along with 

22 the control information at 200 MHz to the lookup engine. The incoming SPI block also 

23 calculates 4-bit Diagonal Interleaved Parity (DIP-4) and reports an error if the calculated 

24 DIP-4 is not equal to the one received with the control word. 

25 The incoming SPI block also sends 2-bit FIFO status information back to the 

26 transmitting device. This FIFO status information is originated from the segmentation 

27 block indicating FIFO congestion conditions for all the ports. The SPI block sends the 

28 FIFO status to the transmitting device at the rate of one port per cycle in a round robin 

29 fashion based on the port calendar sequence. 
30 
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1 Outgoing SPI-4 Interface 

2 The Outgoing System Packet Interface Level 4 (SPI-4) interface transmits 1 6-bit 

3 DDR ATM or Packet over SONET/SDH (POS) data to the receive device (PHY/Switch, 

4 depending on whether employed in the ingress or egress application) and receives out of 

5 band FIFO status information from the receive device. The outgoing SPI interface gets 

6 data and control information from the re-assembly engine, and passes the FIFO status 

7 information to the database. Furthermore, the outgoing SPI-4 interface also interfaces 

8 with the CPU through a software interface. The CPU can program the registers inside the 

9 blocks that comprise the Outgoing SPI-4 Interface. 
10 

11 Features 

— 12 - Supports SPI-4 phase II spec at 10 Gbps rate. 

3 13 . Unpacks the 64-bit data at 200 MHz to 16 bit DDR data at 400 MHz. 

14 - Muxes the data and the control onto the SPI-4 bus. 

)B 15 - Diagonal Interleaved parity generation. 

5 1 6 - Programmable data transfer burst sizes of multiple 1 6 bytes with minimum of 

17 80 bytes, 

y is - Passes the FIFO status info to the database to stop transferring. 

n 19 - Schedules training sequence for data path de-skew. 

^ 20 - Supports maximum 64 ports. 

; "-5 
war 

M- 21 

22 Functional Description 

23 The Outgoing SPI-4 Interface receives 64-bit data and control information from 

24 the re-assembly engine at 200 MHz. The Tx state machine generates 1 6-bit control 

25 words, unpacks the 64-bit data into 1 6-bit data words, and multiplexes the control and 

26 data words onto a 1 6-bit, 400 MHz DDR bus to an external receive device. The Outgoing 

27 SPI-4 Interface also passes the flow control information from the receive device to the 

28 database block within the Maximus chip. See Figure 54. 
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1 

2 Block Description 

3 

4 Tx State Machine 

5 The Tx state machine stores and monitors the length field from the reassembly 

6 block. The length field is decremented on every data transfer from reassembly to SPI. 

7 When the length reaches a threshold, the Tx state machine issues a request command to 

8 read the next port data and control information out of reassembly block. The threshold 

9 level is determined by a value that would allow back-to-back data transfer from 

10 reassembly block into SPI without gap on the data bus. 

1 1 The control word format is common for both incoming and outgoing interfaces. 

12 Refer to the Incoming SPI Interface document for details. 

G 1 3 The state machine also schedules to send the training patterns for the data path de-skew at 

14 least once every pre-defined DATAMAXT cycles. The training pattern is shown in 

15 Figure 55 from cycles 1 through 20a + 1 . The a is the number of repetitions of the data 

16 training sequence. The DATA MAX T and a are conFigured at the startup. 

17 The state transition table is shown in Figure 56. At each state, two 16-bit words 

1 8 and pos_v and neg_v have to be output. And the pos_v and neg_v will be used to 

19 generate a DDR control signal. 1 indicates a control word and 0 a data word. If data_valid 
P 20 is set, there are data to be sent. Otherwise the state machine needs to generate one of the 

21 control words. 
22 

23 Data Parity Generation 

24 This block calculates 4-bit Diagonal Interleaved parity (DIP-4) and sends the 

25 DIP-4 along the control words. The extent of the parity calculation is shown in Figure 

26 57. A functional description of calculating the DIP-4 code is given in Figure 58. 
27 

28 Unpack Engine 

29 This block receives 64 bit data from Reassembly block and samples data at 

30 200MHz clock rate into the SPI out block. See Figure 59. 
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1 The 64-bit data and a data valid signal are received following the control info 

2 read. The unpack engine unpacks the 64-bit input data into 16-bit words. In order to send 

3 16-bit DDR data at 400 MHz to the LVDS I/Os, the state machine has to come up with 

4 two 16-bit words every 400 MHz clock cycle. See Figure 60. 
5 

6 Configuration Register 

7 This block defines the organization of configuration space based on SPI-4 phase 2 

8 specifications. 
9 

10 Port Calendar 

1 1 This block stores the predefined calendar information to identify each port is 

12 ready to send data via the SPI interface. The port calendar contains three fields to indicate 

1 3 the PORT ED, JUMP and VALID. The port ID represents the port number for 

14 transmission of data and receiving of status. The port calendar can store up to 64 ports, in 
2 15 the event that less that 64 ports are used, JUMP indicates the end of calendar. VALID 

Ul 16 indicates if the port is valid or not. If a given port is not valid, the next PORT ID should 

ins? 

M= 17 be used, and the PORT ED marked as not valid should be skipped. 

18 When CPU updates the port calendar after initialization, CPU needs to meet the 

CH 19 following conditions : 

2 20 - CPU can only modify one port at a time. 

J=f 2 1 - When CPU write a port, CPU needs to specify a port entry in the port calendar 

22 between port 0 to 63, and the specific port will be modified. 

23 - When CPU read a port, CPU needs to specify a port entry in the port calendar 

24 between port 0 to 63, and the specific port will be read out. 

25 - CPU needs to stop the traffic for a given prior to modifying the port attribute 

26 (Jump, Valid). 

27 See Figure 61. 
28 

29 Port Calendar State Machine 

30 This block waits for rdy_4_nx_port_id from TX state machine to indicate the 

3 1 transmit state machine is ready to accept another port request. Upon receiving 
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1 rdy_4_nx_port_id, calendar state machine issues the next port ED from the port calendar 

2 to the TX state machine. However, before issuing the next port ID, the status bit is check 

3 to ensure the SPI FIFO status is in the "Hungry" state for the given port. There is a 64-bit 

4 register to retain the value status on a per port basis. The block matches incoming status 

5 arriving into SPI against attributes from Port Calendar to identify the given port traffic 

6 status. 

7 After Port Calendar issues a valid port to the TX state machine, the Port Calendar 



8 State Machine pre-fetches the next Port ID from Port Calendar by performing a read. If 

9 the TX State Machine issues a read before a valid Port ID is inside Port Calendar State 
10 Machine, no_valid_port_id_available is returned to the TX State Machine. 

11 

12 Status Parity Check 

Q 1 3 This block checks 2-bit Diagonal Interleaved parity (DIP-2) and verifies DIP-2 

%3 14 parity bits for signal integrity. Assuming the TSCLK is running at 400 MHz, this block 

^ 1 5 operates at the 400 MHz clock domain and samples the tstat [1 :0] twice every clocks. 
US 16 There are two 65-bit registers to hold the FIFO status. The first 65-bit registers 

[J 1 7 hold the status value before parity check (pre_chk_st) and the second register holds the 

L 1 8 status after parity check (post_chk_st). Once the parity check passes, this block is 

P 1 9 responsible to copy the status from the pre chk st to the post chk st register. If there is a 

2 20 parity error, the error is reported into the control status register (spio_err [0]) by setting 

Q 21 Tb 1 into the bit. This spio err register is self-cleared upon CPU read. 

22 This block sends SPI status from post_chk_st to the Database block. The SPI port 

23 status will be sent in spio_dbs_stt [7:0], and status for port 0-7 should be sent when the 

24 spio_dbs_sync is asserted, and status for port 8-15 should be sent in the following clock, 

25 status 16-23 on the next clock, etc. SPI status spio_dbs_stt [8] should contain the status 

26 for the CPU port. spio_dbs_sync should be derived from the global_sync signal. 

27 When tstat [1:0] is 2'b01, this indicates "Hungry" and transfer of up to 

28 ((MaxBurst2)*(l 6-byte)) blocks or the remainder of what was previously granted 

29 (whichever is greater) may be sent. Since Maximus internal cell size is 64-byte plus 

30 header (up to 16 bytes), after SPI-out provides an indication to Maximus to stop 

3 1 scheduling additional cell to memory, SPI-out needs to allow the remainder of current 
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1 cell to go out of SPI. In other words, reassembly block does not support shipment of 

2 partial cell to SPI, each time SPI request for a cell, SPI must guarantee to be able to 

3 accept one complete cell. Therefore, the MaxBurst2 value should be programmed to 

4 minimum of 5. Once programmed to 5, when the "Hungry" indication is received, the 

5 remaining bytes in a given cell can still be shipped out. To notify Maximus to stop 

6 scheduling the next cell of data is the spio dbs stt bit from SPI-out to Database block. 

7 Consequently, spio_dbs_stt should be set to one when "Hungry" is received for a given 

8 port. 



9 This interface between SPI-out and database drives the status of the 64 ports in 

10 the output SPI and the CPU port. It is a continuous interface. Every 8 cycles of 200MHz 

1 1 the status indication of the 65 ports is updated to a 65 bits register. The first cycle of the 

12 eight is marked with a specific signal (sync signal), 
y 13 Figure 62 shows the interface: 

Sj 14 A synchroniztion FIFO is used between the 400 MHz and 200 MHz clock 

15 domain. 

W 16 This block detects the synchronization pattern from tstat [1:0] by comparing the 



U 1 7 incoming tstatfl :0] against the framing pattern. Also, the port calendar state machine 

L 1 8 checks training sequence to see if there is any skewing on FIFO status and control lines. 

P 19 To check the training sequence for skew, this block needs to implement a counter for 

5 20 internal tstat, called sync [1 :0], which cycles the nominal value of tstat [1 :0] with respect 

P 2 1 to Maximus internal clock. The value of sync is compared against the value of tstate [1:0] 

22 (external) from the SPI pad and the difference between the two pairs of signals 

23 determines if there is any skew of control signals. See Figure 63. 
24 

25 TX FIFO 

26 This block implements a 4x 1 6-bit dual port SRAM to synchronize Maximus and 

27 SPI clock domains. On the incoming side, the write port operates at 200 MHz and pushes 

28 data into the TXFIFO when data_push is asserted by TX state machine. On the outgoing 

29 side, the read port operates at 400 MHz DDR and pops data into the tdat [15:0] when 

30 data_pop is asserted by TX state machine. 
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TX Status FIFO 

This block synchronizes status from 400 MHz to 200MHz domain using 4xl6-bit 
dual port SRAM. 

Memory Description 

Data Structure for Port Calendar 

See Figure 64. 

- Memory type: single-port SRAM 

- Memory configuration: 64 x 8 bit . 

o Port ED: hold port identification value programmed by the CPU. 

o Jump: indicates the end of the port calendar. When Jump is asserted, 
the given location is the last entry on the port calendar. Upon detecting 
the assertion of jump, the port calendar should recycle the port ID 
from the top of the port calendar. 

o Valid: When valid is asserted, it indicates the given port hold a valid 
connection. When valid is de-asserted, the port calendar will skip the 
de-asserted port and go to the next port ID on the port calendar. 

Data Structure for TX FIFO 

- Memory type: dual-port SRAM 
Memory configuration: 4 x 16 bit 

The TX FIFO is used to synchronize data between different clock frequencies. 

Data Structure for Status FIFO 

- Memory type: dual-port SRAM 

- Memory configuration: 4 x 16 bit 

The Status FIFO is used to synchronize data between different clock frequencies. 



51 



AZA-001/2001-P00 



PATENT 



1 

2 Interface 

3 See Figure 65. 
4 

5 Timing Diagram 

6 Timing information between SPI-out and Reassembly, see Figures 66 and 67. 
7 

8 Register Description 

9 Start-up Parameters, see Figure 68. 
10 

11 Software Interface 

□ 12 The software interface programs Outgoing SPI with a CPU. All Outgoing SPI's 

® 1 3 control registers and memories can also be written and read for test purposes through the 

Nl 14 software interface. The following sections list all the control registers and all the 

yj 1 5 Outgoing SPI software interface commands. Please see a dedicated document for a 

J^f 1 6 detailed description of the software interface. 

L 17 

ft* 18 Registers 

P 19 See Figure 69. 

G 20 

"~ 21 Commands 

22 

23 Read Outgoing SPI Start-up Parameters. 

24 See Figure 70. 

25 Write Outgoing SPI Start-up Parameters 

26 See Figure 71. 
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1 Read Outgoing SPI Start-up Parameters 

2 See Figure 72. 

3 Write Outgoing SPI Start-up Parameters 

4 See Figure 73. 

5 Read Outgoing SPI Start-up Parameters 

6 See Figure 74. 

7 Write Outgoing SPI Start-up Parameters 

8 See Figure 75. 

9 Write PORT Parameters after Initialization 

10 See Figure 76. 

1 1 Read PORT Parameters after Initialization 
5 12 See Figure 77. 
^ 1 3 Read PORT Full Parameters 

14 See Figure 78. 

1 5 Write PORT Full Parameters 

16 See Figure 79. 

1 7 Read Control Status Register 

18 See Figure 80. 
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1 Write SPIOJTSTMUX_SEL 

2 See Figure 81. 

3 Read SPIOJTSTMUXSEL 

4 See Figure 82. 

5 Reserved Op-codes 

6 All op-codes other than listed above are reserved. 
7 

8 Principle of Operation 

O 9 The outgoing SPI block receives FIFO status from the Incoming SPL The 

•-f? 

%J 1 0 beginning of the status is found based on the framing sequence. The beginning of the 

^ 1 1 status from SPI interface is mapped to the port calendar to look up the status of each port. 

jj 12 The status has to be synchronized to the internal 200 MHz clock and passed to the 

ft 13 database block, and internal state machine to select a port that is available for data 

^ 14 transmission. Base on the status information, outgoing SPI block requests data from re- 

gt 1 5 assembly block on a given port. Outgoing SPI unpacks the port data from re-assembly, 

? 1 6 delineation data between different ports and check for data error using control 

O 1 7 information from re-assembly. The outgoing SPI block calculates the 4-bit Diagonal 

1 8 Interleaved Parity (DIP-4) and combines those four bits with the control information to 

1 9 form a 1 6-bit control word. At every clock cycle of the 400 MHz clock, the outgoing SPI 

20 unpacks the 64-bit data into 1 6-bit data words and inserts control words by multiplexing 

21 onto 16-bit DDR bus at 400 MHz. If the re-assembly's control FIFO is empty, the 

22 outgoing SPI block sends idle control words onto SPI data bus. During regular interval, 

23 the outgoing SPI also sends training control word to provide skew detection. 
24 
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1 Lookup Engine 

2 This block receives ATM/Packet data and control from the SPI-4 receive interface 

3 block. It looks up a Flow ED and a set of parameters from a hash table in external 

4 SSRAM and forwards the data flow with the parameters to the segmentation engine. 
5 

6 Features 

7 • Supports up to 1M flows. 

8 • Supports up to 64 input ports. 

9 • Operates at 200 MHz/5ns cycle time. 

1 0 • Tables and configuration registers loaded via CPU Interface. 

1 1 • Label extraction is programmable on a per port basis. 
« 1 2 • Accepts packet SOP cells of 64 Bytes only. 

i 13 • Accepts ATM cells of 52 Bytes only. 

1 4 • Direct header stripping is programmable on a per port basis. 

ft 15 • Hash generator creates 4 hash values or slots in parallel from a key using 4 separate 

ffl 16 CRC polynomials as hash functions. 

U 

- 17 • Two bins per bucket or record to minimize collisions. 

if 18 • There are two record memories for up to 8 read operations per a tag lookup. 

Li : 

O 19 • Hash table lookup constrained to resolve within 8 read operations. 

O 20 • Overflow CAM of32X 72 entries for lookups that fail to resolve in 8 read 

21 operations. 

22 • Fully pipelined lookup operations. 

23 • Supports a maximum of two 8 X 72 M SSRAM. The normal size is two 2 X 72 M 

24 SSRAM. 

25 • Supports the following flows with the following headers: 

26 o MPLS with PPP encapsulation. 

27 o LC-ATM that is VPI/VCI. 

28 o Switch headers. 

29 o FIDS. 

30 • Operates in ingress and egress modes. 
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1 

2 Functional Description 

3 See Figure 83. 

4 The lookup engine does label extraction, label hashing and table lookup for each 

5 data transfer that is SOP with a header. The type of traffic is unique to the incoming port. 

6 There will be no more than one type traffic on a port. The default type will get a special 

7 FID. The default FID is stored in a memory and it is specific to the input port. The lookup 

8 engine can strip the switch header of incoming cells. 
9 

10 Block Descriptions 

1 1 See Figure 84. 
12 

Q 13 CPU Hash Generation 

5 1 4 The CPU through the interface can request a hash generation and get a hash index 

.2 15 as a response. It will be used to set or tear down connections. 
W 16 

m 

M= 17 Port Configuration 

L. 1 8 The memory has two uses. It specifies the type of traffic on that port. Also, it has 

19 the Traffic parameters. The CPU either writes the parameters or the logic after lookup. If 

20 the CPU writes the parameters, then they are not changed until the traffic type is. They 

2 1 will be used in default traffic. If the traffic is direct, then the rest of the parameters in the 

22 memory for that port are not used. If the traffic is ENCAP type, then another set of 

23 registers are used. When the port traffic is none of the above, then the parameters values 

24 such as the type and FID are looked up from the record memory for SOP traffic. Then at 

25 the last stage they are driven on the interface bus. At the same time they are written 

26 immediately in the port configuration memory. For none SOP cells, the parameters are 

27 read from the port memory at the last stage and then driven on the bus. This will avoid 

28 data coherency problem in case the cells for the same port are contiguous. For ATM cells 

29 the SOP and EOP cell is the same. 
30 
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1 DATA Coherency 

2 The parameters on none SOP cells are read near the last stage of the data in order 

3 to avoid driving stale values on the interface bus. The CPU accesses to modify the 

4 memory are not tracked. The parameters used are the ones that are lookup or stored in the 

5 port memory. 

6 If the SOP cell has a match on the last lookup, the parameters are written in the 



7 port memory near the end of the lookup cycle. If the next data cell is for the same port, 

8 then the parameters are not written yet in the corresponding memory location. Therefore, 

9 if the cell is not SOP, the port memory is read near the end of the pipeline of the cell 
10 lookup. See Figure 85. 

11 

12 CPU Accesses 

□ 1 3 The CPU accesses are allowed between lookup sequences when the memory bus 

^4 14 is not busy. 

3 15 

y| 16 External SSRAM 

r? 1 7 The Flow ID tables are implemented in off chip rams using 200MHZ, ZBT 

s 

^ 1 8 SSRAMS. These rams provide single cycle access with no 'dead cycle' penalty when 

5 1 9 mixing reads and writes. The table will contain a maximum of 8M entries. There is a 

CI 20 programmable register that will determine the size of the memory. The hash result will be 

0 21 trimmed to a number of bits that is programmable. There are two SSRAM channels 

22 provided. The usual size is 2M entries per channel. The memory can be accessed directly 

23 without hash for a read and write operations. 
24 

25 SRAM INITIALIZATION 

26 The internal as well as the external ram will be initialized to 
27 

28 HEADER Selection 

29 The header depending on the type has different sources. It can be transmitted with 

30 the traffic as in the case of direct or the special type. It can be found in the port memory 

31 as in the case of the default. Last, it can be found in record memory. See Figure 86. 
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1 

2 Hash FIFO and Record Reading 

3 See Figure 87. The data is stored until the port table is accessed. The label is 

4 generated. The data will continue to be clocked in a FIFO until the key is hashed and the 

5 index is calculated. 
6 

7 Memories 

8 Input Port Parameters Memory 

9 Each port will have a parameter entry in that table. The "type" parameter will 

10 determine if the rest of the entries are used. If the type is "000", then the rest of 

1 1 parameters of that memory entry are not used except for the header strip and tag 

12 extraction. The "HD STRP" indicates the number of bytes to strip from the header. The 
Q 1 3 "INT HDR EXTRACT" indicates the number of bytes that represent the header. The 

^ 14 header is at the end of the header stripped that is stripped. If the type is default, then the 

2 1 5 header to strip is zero. If the header to strip is not equal to zero, then the header will be 

W 16 stripped as in the case of the direct. The same argument is for the tag extract. The header 

03 

£1 17 will contain the FID, CLP, EFCI, and the OAM bits. The bits will be extracted and used 

18 in lieu of the look up data. The number of bytes of the header that are stripped and the 

m 19 parameter location is specified in the table. If the type is "001", then it is a default type 

lI 20 and the rest of the parameters of that memory entry are used. If the type is "001", then the 

^ 21 incoming traffic will be treated differently. The traffic is special and it is treated later in 

22 the specifications in detail. Again there are no hashing or lookup operations. If the type is 

23 different than the above three, then the tag is extracted, hashed and the parameters are 

24 looked up from memory. There are 64 entries in the table because the device can support 

25 a maximum of 64 ports. When there is a lookup operation, the parameters that are looked 

26 up are stored in the port memory and are used on the traffic until the next SOP cell. 

27 Infields where the hardware and the software perform the write operations, the software 

28 does it for the default traffic only, the other types of traffic are updated it by the 

29 hardware. See Figure 88. 

30 The CPU writes the data in the port table entries for the default or the direct type. 

3 1 The logic will write the parameters for the port if it the type is not any of the above. The 
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1 SOP cell will trigger the lookup. The rest of the cells on that port will use the table until 

2 the EOP or the next SOP. Then the parameters will change again. 

3 In L2 encapsulation, if the CRC bit is set, then assert the CRC signal to the segmentation 

4 engine. See Figures 89 and 90. 
5 

6 Table Lookup Memory 

7 The index or hashed key provided by the hash generator is used as an address. 

8 The number of bits that are used from the hashed key or index is programmable and 

9 depends on the size of the SRAM. The address is a base one and it is used to access four 

10 entries in the hash table located in external SSRAM. When these locations are read, the 

1 1 hash key is compared to the content of the memory. If the first location does not match, 

12 then there is collision and the second one is compared the same way. The process is 

O 1 3 repeated for all four entries. If there is no match and collisions resulted from the compare 

H 14 operations, then the process is repeated for the second hash result or index and so forth. If 

1 5 at the end there are only collisions, then the CAM is accessed and the process of 

W 16 comparison is repeated. If there is a hit, then in memory location there is Flow LD (FLD), 

u 17 the QOS, and the rest of the control bits. Each hash table entry requires 72 bits. The bit 

L 18 definitions are summarized in Figure 9 1 . The output port number is used to access a 

y 

05 1 9 register. If the register bit corresponding to that port is set, then the EOP cell will have 

lI 20 the error or discard flag set. 
« 21 

22 Hash Data Memory 

23 See Figure 92. The data FIFO is a 15 X 104 bit wide FIFO. The FIFO 

24 compensates for the pipeline delay of the lookup path. It also provides buffering that is 

25 needed when the segmentation engine is not ready to accept any data. The data receiver 

26 block writes the FIFO. The data transmitter block reads the FIFO. The segmentation 

27 engine will assert a ready signal when it can accept data. When the ready is asserted, then 

28 the data is pushed to the segmentation engine. 
29 



59 



AZA-001/2001-P00 



PATENT 



1 Interface 

2 See Figures 93 and 94. When the segmentation engine asserts the 

3 "ALMOST FULL SIGNL", the lookup engine will finish transferring the cell of eight 

4 transfers. Then at the cell boundary, the lookup engine will terminate transmission. At the 

5 same time the lookup engine will signal the SPI interface to terminate pushing data to the 

6 look up engine. The pipe of tag lookup operations is not violated. There are no lookup 

7 operations when the traffic flow is suspended. 
8 

9 CPU Interface 

10 There are signals that are generated by the CPU interface and are used to control 

1 1 the enable and disable signals. See Figures 95 and 96. 

□ 12 

jB 13 Software Interface 

J 1 4 See Figures 97 and 98 . 

^ 1 5 The default value for the memory size is "00 1 " which corresponds to 2 MEG. 
W 16 

1 17 FID Memories Select 

2 1 8 See Figures 99 and 100. The default value for chip select is "00". 
Q 19 

g 20 Register Description 

H= 21 The CPU interface doesn't support burst capability, instead, any read or write 

22 command will terminate with one address location; data bus is still 32 bits. Lookup 

23 Engine block has a fixed size of addresses allocated to it and all accesses to any 

24 internal/external memories or register is done via that specific address space. 

25 This method require the CPU Interface manager to have a mapping of the address spaces 

26 per block so that an incoming command will be redirect to the correct block. 
27 

28 Features 

29 • Databusof32bits. 

30 • Single address access (not a burst access) 
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1 • All external memories and most of the internal memories are accessed via indirect 

2 accesses through registers 

3 • Each block has dedicated addresses for its internal registers of up to 64 registers. 

4 A command drives address and command (Rd or Wr) to the address FIFO and data if it's 

5 a write command. Each block starts it's own CPU access by POPING the 

6 address/command from the FIFO. It means that any CPU transaction starts when the 

7 address/command FIFO is not empty. 
8 

9 Address space 

10 The address space is 64 entries only. The internal registers directly mapped to 

1 1 these addresses and external/internal memories are indirectly mapped. 
n 12 The address spaces per lookup engine are shown in Figure 101 . 

w 13 

S3 14 Direct/Indirect Accesses 

15 A direct access is used to access. If there are internal memories or registers, which 

81 16 are longer then 32 bits, then read or write to these locations will be executed in more then 

3 1 7 one access. For example an entry in internal memory of 50 bits width (or a register with 

1 8 the same length) will be accessed in two CPU commands first access bits [31:0] and then 

O 19 [49:32]. Each register or small internal memories have their own address, which is unique 

H 20 to the entire chip set. 

-^21 An indirect access is used to access internal and external memories. Each block 

22 has a set of registers dedicated for the indirect access. One register contains the address 

23 and the command (Rd, Wr, Init etc). Few other registers contain the data for a write 

24 command. Writing to the COM/ADD register triggers the execution of the command, 

25 once the block completed the execution of the command, the COM portion of the 

26 command is cleared by the block. When a read command occurs the data is written to the 

27 registers and then the CPU reads the registers to get the data. 
28 

29 Registers 

30 See Figure 102. 
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1 

2 Commands 

3 NOP 

4 See Figure 103. 

5 Read LUT Memoryl (72 bits) 

6 See Figure 104. 

7 Write LUT Memoryl (72 bits) 

8 See Figure 105. 

9 Read LUT Memory2 (72 bits) 

10 See Figure 106. 

1 1 Write LUT Memory2 (72 bits) 

12 See Figure 107. 

1 3 Read Port Memory (44 bits) 

14 See Figure 108. 

1 5 Write Port Memory (44 bits) 

16 See Figure 109. 

17 Setup connection command for KEY 

1 8 See Figure 1 1 0. This command sets up a connection by accessing external 

19 memories and initializes the corresponding locations to the proper parameters. It is a 

20 write command. 
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1 Tear-Down connection command for KEY 

2 See Figure 111. This command tears down a connection by accessing external 

3 memories. It is a write command 

4 GET HASH FID 

5 See Figure 112. The response will be in the register. See Figure 1 13. This 

6 command tears down a connection by accessing external memories. It is a write 

7 command. 

8 Read CAM Memoryl (72 bits) 
See Figure 114. 

Write CAM Memoryl (72 bits) 

See Figure 115. 
Read CAM Memory2 (72 bits) 

See Figure 116. 
Write CAM Memory2 (72 bits) 
See Figure 117. 

1 6 INIT ALL MEMORIES 

17 See Figure 118. 

1 8 INIT MEMORY 1 & CAM 1 

19 See Figure 119. 

63 





AZA-001/2001-P00i^ W PATENT 

1 INIT MEMORY 2 & CAM 2 

2 SeeFigurel20. 
3 

4 General commands 

5 The following commands are combinations of the above simple commands. A 

6 user command like this should be broken to several Read or Write commands in the 

7 driver. 



9 Initialized External memories 

1 0 • Write "0" in all Record Memory 1 . 

1 1 • Write "0" in all Record Memory 2. 

12 

1 3 Initialized Internal memories 

1 4 • Write "0" in all Port memory. 
15 

16 Principle of Operation 

1 7 The lookup engine has to perform a port lookup before the FID lookup operation is 

1 8 performed. The memory is initialized and the valid bit is set for that FID. IF the valid bit 

19 is not set, then there will be no match. The lookup engine has the port of incoming traffic. 

20 As a result the following is done: 

21 1 . Read the port parameter entry by accessing the port parameter table. 

22 2. Get the LOOKUP TYPE. 

23 3. Extract the tag. 

24 4. Perform the stripping, hashing, or lookup operations according to the type. 

25 Once the parameter entry is read, then the lookup type is known. If the lookup type is 

26 direct, then the lookup engine will extract the FID, QOS, CLP, EFCI, and OAM if any 
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1 and push all the control and data to the segmentation engine. If the traffic is default, then 

2 the parameters from the port table are used. If the traffic is L2, then the operation is 

3 handled as described in another section. If the traffic type is none of the previous ones, 

4 then the tag is extracted according to the type of traffic and then it is hashed. The result of 

5 hashing will be used to access the memory to get the rest of the parameters if there is a 

6 match. The memory content of the tag is compared to the hash tag. If they are equivalent, 

7 then the rest of the memory content is used for that FID, QOS, CLP, EFCI and OAM. If 

8 the tags do not compare then there is collision and the next hash result is used. There are 

9 four hash generators that will generate 4 indices. Two indices will be used for one 

10 memory and the other two for the second memory. Each record holds two bins. The total 

11 is 8 bins that might collide. If there is no match but collisions, then the overflow CAM is 
^ 12 accessed. If there is collision or no record, then the packet or the cell is marked for CPU 

1 3 and a register is used for the FID. 

1 14 

fl 15 CPU FIDS 

IB:; 

1 16 If the keys stored in the memory locations do not match the hashed key, then the 

u. 

1 7 CAM content will be compared and the if it fails to compare, then the traffic will be sent 

2 1 8 to the CPU. There is a register that holds the FID and the rest of the bits necessary for the 
O 19 data traffic. 

; s 

□ 20 

fr* 21 TAG Extraction 

22 The lookup engine requires the minimum burst size of 1 6 bytes for the key 

23 extraction. The hardware extracts a contiguous number of bytes starting at an offset 

24 according to the traffic type of that port. The type of label to be extracted is determined 

25 by the port number. The location of the label is dependent of the type of traffic and is 

26 hard coded in the hardware. 
27 

28 ATM TYPES 

29 If the type of the traffic is ATM, then the CLP, EFCI, and OAM bits are extracted 

30 and driven to the rest of the device. See Figure 121. Extract the above bits from header 

31 and drive to segmentation. 
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1 

2 TAGS 

3 The following Figures show the data formats for each of the supported protocols. 

4 The specifications are not hard coded. The port table will be used for the header's 

5 location. Each field is 8 bits wide with 12 bits VPI and 16 bits VCI field, for a total of 28 

6 bits lookup tag. This 28 bits lookup tag is located in the 1 st 64-bit word, bits 32 thru 63. 

7 See Figures 122 and 123. The PPP protocol is part of the link layer header. The tag is bit 

8 60:41 of the first 64 bits word. 
9 

10 Port Types 

1 1 The first lookup is to read the port parameters. If the type is "000", then the type 
^ 12 is direct flow. There are no hash or lookup operations. The FID will be extracted from the 
>& 1 3 incoming traffic itself. If the type is "00 1 ", then it is a default traffic. The default 

^ 14 parameters are stored in the table itself. There are no hashing operations or lookup in this 

1 5 case. The third special case is the L2 type. The incoming header will be used to find the 

© 16 encapsulation header. There will be no hash and lookup operations either. The rest of the 

8 * 

1~ 17 lookup types require a hash and lookup operation. The port table will specify the type of 

2 1 8 lookup that the lookup engine needs to perform on the incoming data. The lookup 

O 1 9 operation will assign an FID to the data stream or packet or flow, except in the bypass 

^ 20 type. The FID is used only internally. See Figure 124. 
M- 21 

22 Direct Header Format 

23 See Figures 125-7. The SOP and EOP have to be extracted from the header. The 

24 SPI interface will also assert the SOP and EOP signals. The lookup engine will determine 

25 the set that uses and asserts to the segmentation engine from the type of the packet. In 

26 egress mode, the device uses the header set of SOP and EOP signals depending on the 

27 type. For example, if the switch fabric is a packet switch fabric, then the lookup engine 

28 will use the SPI interface set. If the switch fabric is a cell one, then the lookup engine will 

29 use the set parsed out of the header. When the header is stripped, then the number of 
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1 valid bytes has to be adjusted accordingly. The data has to be packed in 64 bit transfers to 

2 the segmentation engine. 
3 

4 Special Header Addition 

5 If the type is "010", then the flow will be handled differently. The incoming frame 

6 structure will look like the following: 

7 See Figure 128. The CID and the CLASS will become the new FID of the flow. 

8 The rest of the incoming header will be used as the internal header. If the "SPCL LK" is 

9 set, then the encapsulation header is added. If it is not set, then the header is extracted and 

10 no special header is added. The memory will have the structure depicted in Figure 129. 

1 1 The memory location will be read and the length will indicate the number of valid 
O 12 bytes that have to be appended before the data as a special header. The header can be up 
%3 13 to 16 Bytes long. The rest of the bits such as the EFCI, CLP and the OAM are driven, too 
J! 14 as out of band signals. See Figure 130. The byte valid from the SPI interface to the 

^ 1 5 segmentation engine has to be adjusted for the fact that the extra bytes have been added. 

U 1 6 The data has to be packed if the number of byte is not on 64 bit boundary. 

£2 17 

OS 18 Hash Index Generation 

S 19 The extracted label is concatenated with the input port to produce the key. Keys 

P 20 . may be padded out to 32 bits if they are less than 32. The key is hashed in hardware 

2 1 using the irreducible ETHERNET CRC32 polynomial listed below and three irreducible 

22 trinomial polynomials: 

23 1. G(X) = X 32 + X 26 +X 23 + X 22 +X 16 +X 12 +X U + X 10 +X 8 + X 7 + X 5 + X 4 + X 2 + 

24 X'+l; 

25 2. G(X) = X 31 + X 7 +1; 

26 3. G(X) = X 31 + X 6 + 1; 

27 4. G(X) = X 31 +X 3 + 1; 

28 The key is hashed 32 bits at a time, requiring 1 cycle for all keys. See Figure 131. The 

29 type will determine if there is a need for hashing. The tag will be extracted in the first 

30 clock because it is available in the first 64 bits transfer. The key will be formed. The key 

67 



AZA-001/2001-P00 




PATENT 



1 will then be hashed using four different hashing functions. The hashing is done in 

2 parallel. The first hashed key or index will be used as a base register to read two memory 

3 locations. As soon as the first location is read, then the key that is stored in that memory 

4 location is compared to the hashing key. If there is a match, then the read operations are 

5 halted. The content of the memory of the matching key will be used. If there is no match, 

6 then the next location content will be compared and so forth. The second index will be 

7 used to read the third and forth records if the first record of two bins fails to get a match. 

8 Then the third and the forth indices are used for the second memory. If all four indices 

9 fail to get a match, then the CAM is accessed. If the CAM contents do not match, then 
1 0 the traffic is forwarded to the CPU. 

11 

12 Timing Diagrams 

Q 13 See Figure 1 32. The transfer size is assumed to be 64 bytes for up to eight 

^ 14 transfers for SOP SOB data traffic. 
M 15 

y 16 Segmentation 

17 The segmentation engine segments the incoming packets into 52-byte ATM, 48 

* 1 8 bytes AAL-5, or switch cells and stores them in memory as cells of 64 bytes. The 

H 19 segmentation engine adds the necessary number of pad bytes to complete the cell. 
° 20 

O 21 Main Features 



22 - Segments incoming data from SPI-4 on per port basis, supports maximum of 

23 64 ports 

24 - Segments incoming data from CPU interface 

25 - 64K bytes maximum packet 

26 - Segments, maps or de-maps incoming data into either ATM/ AAL-5 cells or 

27 switch cells 

28 - Stores segmented cells in memory as cells of 64 bytes 

29 - Supports the traffic depicted in Figure 133. 

30 - Types of segmentation and applications for various traffic types. 
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Backpressures the scheduler in the egress side from the per flow Q in the 
ingress side via status cells through a serial interface to segmentation engine. 
Calculates CRC-32 for packets segmented into AAL-5 cells. 
Calculates new CRC-32 for L2 header. 

Supports programmable maximum transfer unit up to 16K bytes for packet 
data. 

Each port has a 32-bit counter to count packets coming from that port. 
Each port has a 48-bit counter for data through that port. 
Tags bad packets that they can be discarded later. 
Internal memories to store parameters and data, 
o Dual-port SRAM 

■ Control - 65 x 39 bits to store packet length, and cell byte count 

■ Data - 640 x 64 bits to store data 

■ CRC - 65 x 32 bits to store the partial CRC for each port 

■ Statistic - 65 x 80 bits to store packets and data received for 64 

ports, and one for CPU 

o Single-port SRAM 

■ Buffer - 80 x 7 bits to maintain the free buffer list for data 
SRAM 

■ Queue - 80 x 70 bits to queue information for the complete 
cells 

Operates at 200 MHz 
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1 

2 Functional Description 

3 See Figure 134. There are three input data paths to the segmentation engine: data 

4 from SPI-4, CPU, or serial interface. There are two output data paths from the 

5 segmentation engine: to memory manager, and serial interface to the scheduler. 

6 See Figure 135. The segmentation engine receives data from the incoming SPI-4 

7 during normal operation. Based on the type of application, the segmentation engine 

8 segments, de-maps, or maps the incoming data to either ATM/AAL-5 or switch cells. 

9 As shown in Figure 133 there could be five different types of segmentation (as opposed 

10 to types of application) performed by the segmentation engine. The following explains 

1 1 the operation of the segmentation engine for each of these types. 
12 

13 TYPE 1 Segmentation 

14 See Figure 136. Type 1 segmentation is for application type 0, 4, 8, 9, 10, 11,12, 
H 15 and 14. The segmentation engine receives the 64-byte cells from the look up engine and 
^ 16 queues the cells to the output control block when the entire cell has been received. 

|n 

5 1 8 TYPE 2 Segmentation 

^19 See Figure 137. Type 2 segmentation is for application type 1 and 5. The 

O 20 segmentation engine receives the 64-byte cells from the look up engine and queues the 

Q 21 cells to the output control block when the entire cell has been received. The cells come in 

22 as 52 bytes ATM data with 12 bytes of pad and output as 48 bytes of AAL5 data with 16 

23 bytes of pad to the memory manager block. The segmentation engine removes the 4 bytes 

24 ATM header and appends 4 bytes of pad to the end of the cell. 
25 

26 TYPE 3 Segmentation 

27 See Figure 138. Type 3 segmentation is for application type 2 and 13. The 

28 segmentation engine receives the 64-byte cells from the look up engine and queues the 

29 cells to the output control block when the entire cell has been received. The cells come in 

30 as 64 bytes packet data and output as 48 bytes of AAL5 data with 16 bytes of pad to the 

3 1 memory manager block. The segmentation engine segments the incoming data into 48 
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1 bytes, appends 16 bytes of pad, and passes the control information for the completed cell 

2 to the output control block. CRC-32 is calculated when the cell is being sent out to the 

3 memory manager block. 
4 

5 TYPE 4 Segmentation 

6 See Figure 139. Type 4 segmentation is for application type 3 and 6. The 

7 segmentation engine receives the 64-byte cells from the look up engine and queues the 

8 cells to the output control block when the entire cell has been received. The cells come in 

9 as 64 bytes packet data and output as 64 bytes of AAL5 like data to the memory manager 

10 block. The segmentation engine segments the incoming data into 64 bytes and passes the 

1 1 control information for the completed cell to the output control block. The segmentation 

12 engine calculates CRC-32 and appends an AAL5 like trailer at the end. However, CRC- 
Q 1 3 32 is calculated when the cell is being sent out to the memory manager block. 

H 14 

[JJ 15 TYPE 5 Segmentation 

W 16 The segmentation engine actually is not segmenting data in type 5. When the 

U 17 ingress segmentation engine receives a status cell from the egress PFQ through the serial 

L, 18 interface, it queues that status cell to the memory manager like any other cells. The status 

05 19 cell is sent through the switch to the egress segmentation engine, and the egress 

P 20 segmentation engine sends the status cell to the ingress scheduler through a serial 

P 21 interface. See Figure 140. The serial input and output data paths provide a way for the 

22 per flow Q on the egress side to back pressure the scheduler on the ingress side from 

23 sending data. 
24 

25 CPU injected packets 

26 The CPU can inject data through software command as described later in section 

27 1 .6.2. When the segmentation engine detects the CPU inject packet command it sends the 

28 cell out in the next control pop from the memory manager. However, there can only be no 

29 more than one CPU cell sent within 64 cells time. The segmentation engine starts a cell 

30 counter whenever it sends a CPU cell, and no more CPU cells will be queued out until the 

31 counter reaches 64. 
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As shown in Figure 141, the CPU can write and read the internal registers inside 
the segmentation engine through the CPU interface. The CPU can also input packets into 
the segmentation engine through the CPU interface. 
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1 

2 Block Description 

3 See Figure 142. 
4 

5 Description of INPUT CONTROL block 

6 In normal operation, when the segmentation engine has room in its FIFO for 

7 incoming cells it asserts a FIFO not full signal to the look up engine and waits for a valid 

8 signal from the look up engine to indicate there is valid control information on the bus. 

9 When there are valid controls, the input control block performs read from various internal 

10 memories: 

1 1 a) Reads from the FBL SRAM to get two pointers to the data SRAM, 

12 b) Uses the port address to read the SEGTBL SRAM to get the per port parameters. 

1 3 c) Uses the port address to read the STATISTIC SRAM to get the statistics for that 

14 port, 

1 5 The per port parameters (SOP, EOP, etc.) from the bus are compared with those from 

1 6 the SEG_TBL, and error conditions are set accordingly. The port address from the bus is 

1 7 compared to the CUR_PH), which stores the last port address processed, to see if the 

1 8 incoming burst is from the same port address. If so, it means the data burst is the 

1 9 continuation to the one previously processed, and the control information in the internal 

20 registers are used instead of those read from the memories. Otherwise, it means the 

21 incoming data burst is for a new port address and new controls are to be used, and the 

22 controls for the last processed one are to be stored before the new ones are loaded. 

23 Detailed information is discussed later in the timing flow. 

24 The input control block has two sets of control parameters, one for the last processed 

25 port address, i.e. the current parameters, and one for the port address from the look up 

26 engine that comes with the valid signal. 

27 The controls, statistics, and the two pointers are then passed to the data block with a 

28 start signal to. start the data state machine. 

29 The input control block also receives FIFO status from the FBL and QUEUE blocks 

30 and generates the FIFO not empty signal to the look up engine. 
31 
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1 Description of FBL block 

2 The FBL (Free Buffer List) block keeps track of the open entries in the data 

3 SRAM. After power on reset, the CPU needs to write the init bit in the segmentation 

4 register to start the self- initialization of the free buffer list. The self-initialization takes a 

5 little over 80 cycles, and no data is allowed through the segmentation engine block until 

6 the initialization is done. 

7 The FBL block receives the read strobe from the control block. For each read 

8 strobes the FBL block reads two locations from the FBL SRAM and passes the two 

9 pointers back to the control block. 

1 0 The FBL block receives the write strobe and the return pointer from the data 

1 1 block. It writes the return pointer to the write address when there is no read pending. 

12 The FBL block generates a full status to the control block if there are no more free 
n 1 3 entries in the data SRAM. 



S 14 

SI 1 5 Description of FBL SRAM block 

y 16 This is an 80 x 7 bits single-port internal SRAM to store pointers to the free 

P? 1 7 memory blocks in the data SRAM. The CPU needs to set the init bit to start the self- 

s 18 initialization after reset. 

m 19 

Q 20 Description of SEG_TBL SRAM block 

hsas 

□ 21 This is a 65 x 19 bits single-port internal SRAM to store the parameters for the 64 

^ 22 ports plus CPU. This RAM is accessed twice by the control block for each cell comes in 

23 for a particular port, one read to retrieve port parameters and one write to return 

24 parameters for that port. 
25 

26 Description of STATISTIC SRAM block 

27 This is a 65 x 80 bits dual-port internal SRAM to store number of packets (32 

28 bits) and data bytes (48 bits) received on the 64 ports and CPU interface for statistic 

29 purpose. 
30 
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1 Description of DATA PIPELINE block 

2 The DATA PIPELINE block delays the incoming data and byte valid to match the 

3 delay for control signals through the INPUT CONTROL block. 
4 

5 Description of DATA block 

6 The data block receives the control information from the input control block and 

7 decodes the types of application to be performed by its input state machine. The input 

8 state machine starts when the data block receives the start signal from the control block. 

9 The input state machine has five different branches based on the segmentation types 

1 0 decoded from the application types. For each of the branch the input state machine 

1 1 performs the following operations and writes the data into the data SRAM: 

12 a) Add 8 bytes of pad 
□ 13 b) Add 16 bytes of pad 

14 c) Segment incoming data into 48 bytes and add 16 bytes of pad 

^15 d) Segment incoming data into 64 bytes 

y 16 e) Pass through the 64 bytes 

5 1 7 The main function of this input state machine is to accumulate a 64 bytes cell for each 

1 8 port address. When there is a completed cell, the input state machine generates a control 

19 signal to queue the control information for that particular cell into the QUE FIFO in the 

20 queue block. At the same time, the data block writes the statistics back into the STAT 

2 1 SRAM and sends the parameters back to the input control block. 

22 There is a data byte counter which increments for each incoming data bytes to count 

23 for the 64 bytes cell. For a cell with EOP in it, the counter value is sent to the memory 

24 manager indicating the number of bytes valid in the EOP cell. 

25 For AAL5 or like data cell, the data block also check to see if a new cell is needed for 

26 the trailer. If needed so, the data block writes the data/trailer to the buffer entry indexed 

27 by the second pointer. Otherwise, the data block returns the second pointer to the end of 

28 the free buffer list. 

29 There is also an output state machine, which starts when the queue block requests to 

30 de-queue from the data block. This output state machine receives the pointer to the data 

3 1 SRAM and the control information to be processed on the cell. The output state machine 
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1 reads from the data SRAM with the buffer pointer and output the data to the memory 

2 manager. When the data block finish sending data to the memory manager it returns the 

3 pointer back to the free buffer list. 

4 If the control information indicates a need to generate CRC-32, the data block reads 

5 the CRC SRAM with the port address from the control information and load the partial 

6 CRC into the CRC-32 engine. At the end of the cell and not EOP, the data block writes 

7 back the partial CRC into the CRC SRAM. If the control indicates a need to calculate 

8 new CRC for the L2 header, the output state machine replaces the existing CRC bytes 

9 from the packet with the ones calculated with the new L2 header, and the new values are 

10 sent through the CRC engine to calculate for the AAL5 trailer. 

1 1 There are two statistic counters in the data block, one for the number of packet 

12 received and the other for the number of data received. These statistics are kept on a per 
9 13 port basis, where the data block load the information from the STAT SRAM with the 
H \A load signal from the input control block. 

J 15 

W 16 Description of DATA SRAM block 

S3 

U 1 7 This is a 640 x 64 bits dual-port internal SRAM to store the incoming data as the 

%~ 1 8 segmentation engine completes segmenting them into 64-byte cells. The memory is 

© 19 allocated into 80 blocks with each block has eight entries, total of 64 bytes per block. 

1 ;Z 

M= 20 

2 21 Description of CRC_GEN block 

22 This block generates the CRC that covers the PDU (Protocol Data Unit) in the 

23 AAL-5 format. The polynomial used is CRC-32 as follows. 

24 G( X ) = x 32 + x 26 + x 23 + x 22 + x 16 + x 12 + x 1, +x ,0 + x 8 + x 7 + X 5 + x 4 + x 2 + x+l 
25 

26 Since the internal data bus has 64 bits, the CRC-32 is calculated in two portions: 

27 a) CRC-32 engine with 64-bit data input 

28 b) CRC-32 engine with 32-bit data input 

29 From the beginning of the data, CRC-32 engine with 64-bit input data bus is used 

30 all the way to the end of the data, i.e. right before the trailer for AAL5. Then it loads the 
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remainder from the CRC-32 engine with 64-bit input data bus to the CRC-32 engine with 
32-bit input data bus and calculates the CRC with the four MSB bytes of AAL5 trailer. 
The CRCGEN block reads the partial CRC with port ID as the address to the CRC 
SRAM. The CRC_GEN block loads the partial CRC into the CRC engine and starts 
calculating as the data are going out to the memory manager. When the cell is sent to the 
memory manager block, the CRC_GEN block writes back the partial CRC into the CRC 
SRAM, again using port ID as the address. At the end of a packet, the CRC is muxed out 
as part of the AAL-5 trailer. For L2 header, CRC is calculated for the packet and is used 
to replace the CRC bytes in the packet before calculating the CRC for the AAL5 trailer. 

Description of CRC SRAM block 

This is a 65 x 32 bits dual-port internal SRAM to store partial CRC for each port. 
The data block reads and loads the partial CRC if the traffic type requires CRC-32 
calculation. The data block writes back the partial CRC when the completed cell is sent to 
the memory manager. 

Description of OUTPUT CONTROL block 

The OUTPUT CONTROL block sends the available signal to the memory 
manager to indicate that there are completed cells available to transfer. The memory 
manager replies with the control pop signal if it is ready to receive the completed cells. 
When there is control pop from the memory manager, the output control block reads the 
QUEUE FIFO for the control information for that cell and passes them to the data block. 
The data block performs read to the DATA memory as described in the data block above. 
The output control block sends the control information lining up with the output data. 
The output control has to work in compliance with the global sync, it only asserts the 
available signal in second clock slot from the global sync (see detailed timing in later 
section). The memory manager is to responds in the third clock slot, and the output 
control block generates the pop signal to the QUEUE FIFO in the following clock. The 
output control block also generates the read signal to the data memory in the same clock 
slot. 
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1 If there is status cell pending, the output control block sends the status cell using 

2 one of the regular port accesses from the global sync, and the skipped cell takes over the 

3 CPU access during that particular global sync. This means CPU access is skipped for the 

4 global sync that has a status cell to send. The output control block returns to its normal 

5 port access in the following global sync. 
6 

7 Description of Q_FIFO SRAM block 



8 This is an 80 x 70 bits single-port internal SRAM to store control information for 

9 cells queued to the memory manager block. 
10 

1 1 Description of CPUJNTF block 

12 The CPUJNTF block is to interface with the external CPU. It supports CPU 



□ 13 direct access to the internal registers and indirect access to the internal memories. CPU 

14 can also injected packet data through this interface. The CPU can inject packet/data 

N 1 5 through use of CPU inject data command, however, it can only injects a 64 bytes cell one 

■ ft 

ft 16 at a time. As soon as there is a complete cell, the CPU_INTF block asserts a signal to the 

P 17 output control block to output the cell in the CPU access. 
M 18 

% 19 Description of PFQ_SEG INTF block 

O 20 The PFQ_SEG INTF block receives data from the serial interface between per 

Q 2 1 flow Q and the segmentation engine. The PFQ_SEG INTF block shifts the serial data into 

^22 a 64-bit shift register and load the output of the shift register into an internal register. The 

23 control information is extracted from the serial data. When the PFQ_SEG INTF block 

24 receives the entire status cell, it queues the completed cell into the queue FIFO. 
25 

26 Description of SEG_SCH INTF block 

27 The SEG_SCH INTF block receives data from the data block and outputs the data 

28 through a serial interface to the scheduler block. When the segmentation engine receives 

29 a status cell from the incoming SPI-4 interface, the data block outputs the data into the 

30 SEG_SCH INTF block instead of the data SRAM. As soon as the complete cell is 
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received, the SEG_SCH INTF block shifts the cell out the serial interface to the 
scheduler. 

ERROR CONDITIONS 

Several errors can be detected in the segmentation engine, and the error packet 
will be marked error by the SEG_ENGINE. The different errors are in Figure 143. These 
errors are reported in the status register. There are individual bits in the register to mask 
the individual errors from generating interrupt to the CPU. 

Memory Description 

Summary of Internal Memories: 

See Figure 144. 

Data structure for SEG_TABLE 

See Figure 145. The SEG TABLE is a 65 x 39 bits internal SRAM addressed by 
the port ID (PID). The segmentation table has 65 entries, one for each port and CPU. 



BAD 
SOP 
EOP 

Cell Byte Count 
PTR : 
Packet Length 



CLP 
EFCI 

CALC_CRC 
OAM 



This bit is set if detects error condition and needs to discard 
This bit is set if start of packet is received for the port 
This bit is set if end of packet is received for the port 
This field keeps track of the byte count received for the 64-byte cell 
The start address to the cell in the data SRAM 
: This field keeps track of the total length for the incoming packet for 
the port 

Cell Loss Priority, valid only with ATM traffic 
EFCI, valid only with ATM traffic 
Calculate L2 CRC. 
indicate OAM cell. 
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Data structure for FREE_BUFFER 

See Figure 146. The FREEBUFFER is an 80 x 7 bits internal SRAM that used 
to store the pointer to the free data block in the data SRAM. The segmentation engine 
takes a pointer from the top of this table to receive the incoming data. When the memory 
manager gets a complete cell off the queue the pointer to that cell is return to the end of 
this table. 

Data structure for Q FIFO 

See Figure 147. This 80 x 70 bits internal SRAM that stores the queue of 
complete cells and the information for each queued cell. 



FID 

TYPE 

BAD 

SOP 

EOP 

CELLLEN 
PTR 

PKTLEN 

QOS 

0_PORT 

CLP 

EFCI 

CALCCRC 
OAM 



Flow ID of the incoming data for the port. 
Traffic type of the incoming data for the port. 
Mark the packet to be discarded. 
Mark the start of the packet. 
Mark the end of the packet. 

Indicate the number of valid bytes in the 64-byte cell. 
The start address to the cell in the data SRAM. 
Length of the packet, valid only for packet traffic. 
Quality of Service. 
Output port number. 

Cell Loss Priority, valid only with ATM traffic. 
EFCI, valid only with ATM traffic. 
Calculate L2 CRC. 
indicate OAM cell. 



Data structure for DATA SRAM 

See Figure 148. The DATA SRAM is to store the incoming data in 64-bytes 

block. 
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1 Data structure for CRC_TABLE 

2 See Figure 149. The CRC Table is a 65 x 32 bits dual-port internal SRAM 

3 addressed by the port ID (PID). Each port has an entry to this SRAM to store the partial 

4 CRC generated for the AAL-5 traffic. 
5 

6 Data structure for STATISTIC 

7 See Figure 150. The Statistic SRAM is a 65 x 80 bits internal SRAM addressed 

8 by the port ID (PID). Each port has an entry to this SRAM to store the number of packets 

9 and data received for that port. Statistics for the CPU interface is in the last entry. The 

10 CPU interface accesses the content of this SRAM. 

1 1 PKT RCVD : Number of packets received for each port 

12 DATA_RCVD : Number of data bytes received for each port 

" 13 
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1 

2 Interface 

3 

4 Interface Ports 

5 System 

6 See Figure 151. 

7 Segmentation O Lookup 

8 See Figure 152. 

9 Segmentation Memory Manager 

10 See Figure 153. 

1 1 Segmentation PFQ 

12 See Figure 154. 

13 Segmentation SCH 

14 See Figure 155. 
S| 15 Segmentation CPU Interface 
f~: 16 See Figure 156. 

S 17 

3 18 Timing Diagrams 

2 19 See Figure 157. 

0 20 

P§ 21 Memory Access 

N 1 22 See Figure 158. The bolded "Rd" indicates that there are data dependencies for 

23 these reads. The current PID being processed is compared with the incoming one, the 

24 segmentation uses the current per port parameters if they are equal. Otherwise, it uses 

25 those read from the memories. 



82 



AZA-OOl/2001-POi 




PATENT 



1 

2 Registers Description 

3 

4 Registers 

5 See Figure 159. 
6 

7 Commands 

8 Summary of commands 

9 See Figure 160. 
10 CPU Inject Packet 

5 11 See Figure 161. 

Ni 

u « 12 Read SEG FBL memory 

FJ 13 See Figure 162. 

Sew 

0.14 Write SEG FBL memory 

; z : 

Mb 15 See Figure 163. 

? " 16 Read SEG CRC memory 

17 See Figure 164. 

1 8 Write SEG CRC memory 

19 See Figure 165. 

20 Read SEG STATISTIC memory 

21 See Figure 166. 
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1 Write SEG STATISTIC memory 

2 See Figure 167. 

3 Read SEG QUEUE memory 

4 See Figure 168. 

5 Write SEG QUEUE memory 

6 See Figure 169. 

7 Read SEG DATA memory 
~ 8 See Figure 170. 

H 9 Write SEG DATA memory 
5 10 See Figure 171. 

n 11 Read SEG TBL memory 
S 12 See Figure 172. 

jest 

M= 13 Write SEG TBL memory 

14 See Figure 173. 
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1 

2 Principles of Operation 

3 The segmentation engine has three different sources for its incoming data: SPI-4, 

4 CPU interface, and status cells from per flow Q. The control information for the complete 

5 cells are queue into the queue FIFO, and are output to the memory manager following 

6 global sync scheme. If there is a status cell from per flow Q, it will be output in the next 

7 available cell slot, and the CPU slot will be used to compensate for the port that its slot is 

8 taken by the status cell. The segmentation engine resumes to the global sync in the 

9 following global sync cycle. 

10 In normal operation, the segmentation gets its data from the SPI-4. If the FIFO 

1 1 full signal is not asserted, it means the segmentation engine is ready and has room in its 

12 FIFO to receive data from the SPI-4. The look up engine then sends the control 

Q 1 3 information together with the data to the segmentation engine. For the first data cell of a 

SJj 14 packet for a particular port the segmentation engine sets the SOP bit for that port in the 

^ 1 5 segmentation table, writes data into the data FIFO, increments both the packet length and 

hj 16 cell count as data come in. The data state machine keeps receiving data from the look up 

P 1 7 engine until the end of data burst, complete cell size, end of packet, or if there is data 

h 1 8 coming from the CPU or serial interface. 



^19 If it is the end of data burst only, the segmentation engine loads the packet length 

O 20 and cell count back into the segmentation table for that port. The segmentation engine is 

Q 2 1 now ready to receive the next data burst. 

^ 22 If the segmentation engine has a complete cell size, it queues the cell by writing 

23 the cell information into the Q_FIFO. The segmentation engine resets the cell count, 

24 loads the packet length back into the segmentation table. Again, the segmentation engine 

25 is now ready to receive the next data burst. 

26 If it is the end of a packet, the segmentation engine checks to see if the traffic type 

27 is for packet to AAL-5. If so, the segmentation engine checks to see if another new cell is 

28 needed for the AAL-5 trailer. The segmentation engine then queues the completed cell(s) 

29 into the QJFIFO, and then resets the entire entry for that port in the segmentation table. 

30 For packet data to AAL-5 cells, the CRC GEN calculates the ATM CRC-32 when the 

3 1 cells are going out to the memory manager. The partial CRC is stored on per port basis, 



85 



AZA-001/2001-P00 



PATENT 



1 and the partial CRC is loaded back into the CRC generation engine for each cell queued 

2 out the particular port. 

3 The segmentation engine can process multiple open ports. For each port ID it 

4 reads from the segmentation table the parameters for that port, receives data for that port, 

5 stores the parameters back into the table, read the parameters for a different port from the 

6 table and so on. 

7 The segmentation engine queues a completed cell into the Q_FIFO and asserts the 



8 SEG_MEM_AV AIL ABLE signal to the memory manager block. The memory manager 

9 block pops the Q_FIFO to get the information (FID, PID, etc.), and the segmentation 

10 engine reads the data from the data SRAM using the pointer from the Q_FIFO. The 

1 1 segmentation engine sends the data to the memory manager with timing according to the 

12 global sync. After the segmentation engine sends the completed cell to the memory 
O 1 3 manager, it returns the pointer to the FBL. 



H 1 4 If the incoming data is from the CPU interface, the segmentation engine receives 

^ 1 5 data from the CPU FIFO, strips the header from the cell and extracts the control 

■J3 — 

y 1 6 information from the header. The segmentation engine then waits to complete the cell and 

^ 17 queues the completed cell to the Q_JFIFO as in normal operation. 

s 18 If the incoming data is from the serial interface from per flow Q, the segmentation 



m 19 shifts the serial data in, packs to 64-bit bus and store in an internal FIFO. The header of 

?f 20 the incoming cell is stripped and control information such as flow ID and traffic type are 

G 21 extracted from the header. The segmentation engine waits to complete the 64-byte cell 

a a 

*~ 22 and queues it into the control FIFO (Q_FIFO). When the segmentation engine detects a 

23 status cell as the traffic type coming through the SPI interface it queues the complete cell 

24 into a separate FIFO in the SEG_SCH_INTF block and send the data out in a serial 

25 format to the scheduler. 
26 

27 Memory Manager 

28 

29 Features 

30 The Memory Manager (MM) block manages all the data and control traffics 

3 1 from/to the Segmentation, Per Flow Queue, Reassembly, and external SDRAM (or 
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DRAM). Input cells (data) are sent from the Segmentation block to the MM block. The 
ATM cells and packets are sent to the Memory Manager block in the "internal cell" 
format. Internal cell size is determined at the power CPU of the chip (i.e. programmable). 
Figure 175 depicts the top level interface of the Internal Memory Manager block with 
others. 

The Memory Manager block has several main features listed as follows: 

1 . Support up to 5 12 Mbytes of external storage Memory using DDR- 
SDRAM. 

2. Cache the incoming "bypass" cells so that they can be transferred to the 
Reassembly block with minimum latency (without going through the 
external SDRAM path). 

3. Act as a buffer for data going from the Segmentation block to the external 
SDRAM. 

4. Act as a buffer between the incoming burst from the external SDRAM and 
the output to the Reassembly block. 

5. Receive Cell Data and Control Information from the Segmentation block 
and initiate En-queuing process. 

6. Receive De-queuing information (such as BID, FID) from the Per Flow 
Queue, get requested data from either the Internal Memory or the external 
SDRAM, and then send them to the Reassembly block along with other 
related information. 

7. Interface with the Free Buffer List (FBL) inside the Per Flow Queue 
block: 

a. Send En-queuing request to FBL to receive a BID indicating the 
En-queuing cell's data location in DDR-SDRAM. 

b. Send De-queuing BID to FBL to release the location pointed by 
the BID. 

8. Manage the processes and orders of reading from the SDRAM and writing 
to the SDRAM to maximize the overall bandwidth of the SDRAM. This 
relates to the strategy of selection of banks of memory to read and write to 
reduce the overhead penalty. 

87 





1 




2 




3 




4 




5 




6 




7 




8 




9 




10 




11 




i ? 

1Z 


p 


■s 


1 1 

I J 




14 


i 


15 








16 








17 


w 


18 


test 


19 




20 




21 




22 




23 




24 




25 




26 




27 




Zo 




29 




30 




31 



AZA-001/2001-P00 



PATENT 



1 9. CPU Interface: All memories of the Memory Manager can be accessed 

2 (read/write) by the microprocessor through the CPU interface: 

3 a. Internal Memory 

4 b. CAM, Internal Memory List (IML), Free Internal Memory List 

5 (FIML) 

6 c. external SDRAM (by Queuing CPU read/write requests) 

7 10. Frequencies: 200 Mhz 

8 See Figure 174. 
9 

10 Functional Description 

1 1 Refer to Figure 174 for the top level block diagram of the Memory Manager 



12 (MM) block and Figure 175 for the block diagram of the Internal Memory Manager 

^% 13 (IMM). The incoming cells are sent to the Internal Memory Manager block by the 

^ 14 Segmentation block along with other information for the En-queuing process. After 

15 getting the En-queuing BID from the Free Buffer List block, the IMM synchronizes the 

JJf 16 BID with all the information from the Segmentation block and then send them to the Per 

N= 17 Flow Queue block for En-queuing. 

p 18 Incoming cells are temporarily stored in the Internal Memory (internal SRAM) 

^ 19 that is controlled by the Internal Write Manager block. If cells are specified as "bypass" 

M= 20 indicated by the Type input, they stay in the Internal Memory and then they are sent to 

P 21 the Reassembly block directly during the De-queuing process. If incoming cells are 

22 specified as "non-bypass" indicated by the Type input, they are temporarily stored in the 

23 Internal Memory, and then moved to the external SDRAM. The moving of all "non- 
24 bypass" data is handled by processing all the SDRAM write requests with En-queuing 

25 BED/IMBED inside the SDRAM Write Request Queues block. During the De-queuing 

26 process, if some of these cells are called to be de-queued (i.e. CAM Hit) before they are 

27 sent to the SDRAM, their status will be changed in the CAM and they will stay in the 

28 Internal Memory instead of moving out to the SDRAM. These cells will be sent to the 

29 Reassembly block directly. Right after a cell is taken out of the Internal Memory, the 

30 CAM and the Free Internal Buffer List are Updated immediately. 
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1 During the De-queuing process, the Per Flow Queue interfaces with the Internal 

2 Memory Manager block with a De-queuing BID. The MM will check in the CAM to see 

3 whether the requested cell (data) is inside the Internal Memory (SRAM) or inside the 

4 external SDRAM. If CAM indicates CAM Hit and the requested data is a "bypass" type, 

5 it means that the requested cell is inside the Internal Memory. In that case, the Memory 

6 ID Manager (MIM) will issue a command to the Internal Read Manager to read the CAM 

7 hit data out of the Internal Memory; the IMBID (internal buffer ID) will be provided to 

8 the Internal Read Manager. The MIM also modifies the CAM table so that this specific 

9 data will not go to the external SDRAM (it will be marked as "bypass") and it signals the 

1 0 De-Queue Request Manager block not to send a read SDRAM request to the SDRAM 

1 1 Read Request Queues block. 

_ 12 During the De-queuing process, if the De-queuing BID can not be found inside 

5 1 3 the CAM, it means that the requested data is inside the external SDRAM. In this case 

n 14 MIM will signal the De-Queue Request Manager block to send a read SDRAM request to 

^15 the SDRAM Read Request Queues block with the BID and IMBID (provided by the 

I 1 6 MIM block from the Free Internal Buffer List). The SDRAM Read Manager block takes 

1 7 read requests from the SDRAM Read Request Queues and processes them to through the 
Q 1 8 SDRAM Manager (outside of the Internal Memory Manager block). Read-back data from 
| 19 the SDRAM will go through the SDRAM Read Manager block with the data itself, the 
£ 20 BID and the IMBID assigned to it previously. The data and IMBID will go to the Internal 
C 2 1 Write Manager which will open the gate for the data to be stored inside the Internal 

22 Memory block at the IMBID location. As data enters into the Internal Memory, the CAM 

23 table will also be Updated properly (it will be marked as "bypass"). The Internal Read 

24 Manager then will schedule the data to go out to the Reassembly properly based on the 

25 order of the De-queue requests came in previously (stored in the De-queue Queue block). 

26 Microprocessor can access (read/write) the external SDRAM through the CPU 

27 data bus for testing and diagnostic purposes. The access can be random and data are not 

28 managed or shaped. The CPU Interface block inside the Memory Manager sends read or 

29 write requests from/to the SDRAM Read Request Queues and the SDRAM Write 

30 Request Queues, respectively. The CPUID/BID will be queued along with the 

3 1 IMBID/BID. The SDRAM Read Manager and the SDRAM Write Manager will send 
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these information and associated data to the SDRAM BW Optimizer based on the bank 
selection strategy inside it. Data coming back from the SDRAM is routed properly to the 
CPU Interface block based on the returned CPUBID/BID information. 

Block Description 

Internal Memory Manager 

Refer to Figure 175 for the block diagram of the Internal Memory Manager. 



1 0 Internal Memory Block 

This is an internal dual-port SRAM memory. It can run at 200 Mhz with 5 ns 
access time. The input data bus width is 128 bits and the output data bus width is also 128 
bits. The input data, IMBID and control signals are provided by the Internal Write 
Manager block. The output data go to the Internal Read Manager in response to the read 
request along with IMBID provided by the Internal Read Manager block. 

The size of the SRAM is currently defined as 64 Kbytes (= 4K x 128 = IK of 64- 
byte cells). 



19 Memory ID Manager (MIM) Block 

This block consists of a CAM, a Free Internal Memory List (FEVIL), an Internal 
Memory List (EVIL), and a MIM (Memory ID Manager) Controller. Refer to Figure 3 and 
Figure 4 to understand the mechanism of the CAM, FIML, and the EVIL blocks. FEVIL 
and EVIL are internal SRAMs. The CAM contains the BED values. The MIM Controller 
block manages all the incoming commands to Update the CAM, FIML, and EVTL blocks. 
It also arbitrates all the Update commands from different blocks as they can happen at the 
same time. 

Free Internal Memory List (FIML): 
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1 This list contains IK entries (i.e. to support 1024 cells inside the Internal Memory 

2 (SSRAM)). The content of each entry is the IMBID that is an address points to the 

3 Internal Memory List (IML) to get IMBID and status. It also corresponds to the output 

4 address of the CAM. The FIML contents are initialized from 0 to 1023 when an 

5 initialization command is received from the CPU interface. 

6 The FIML has 2 pointers: TakePtr and ReturnPtr. TakePtr and ReturnPtr are 

7 initialized at the top of the List, ReturnPtr points to location 0 and TakePtr points to 

8 location 1 . TakePtr points to a free IMBID value. TakePtr is incremented by 1 every time 

9 an IMBID is taken. ReturnPtr points to a not-available IMBID value. Every time a 

1 0 location is returned to the FIML, a new (i.e. returned) IMBID value will be written into 

1 1 the location that ReturnPtr currently points to. Now that location represents a free cell 
□ 1 2 with the returned IMBID points to a free cell inside the Internal Memory. ReturnPtr is 
m 13 also incremented by 1 every time an IMBID is returned. 



"^5 

N! 1 4 When TakePtr and ReturnPtr reach above the value of 1 023 , they will be rolled 

m 1 5 over to the value of 00. TakePtr and ReturnPtr will never be equal. If (TakePtr - 

J 1 6 ReturnPtr) is equal to - 1 , then the BVI is full and the IMFull bit of the 

17 Mem_Status_0_reg will be set. If (TakePtr - ReturnPtr) is equal to 1 , then the IM is 

Jp 18 empty and the EVMEmpty bit of the Mem_Status_0_reg will be set. The status of Full-1 is 

H 19 also recorded in the IMFulM bit. 

Q 20 The CPU interface should be able to write and read to/from the FIML memory: 

f " 21 1). CPU_Wr_FIML {input: Adr, Wr_value}: 

22 This command allows the CPU interface to write to the FIML table where Adr is the 

23 address value (range from 0 to 1 023) and Wr_value is the content that we are going to 

24 write to. Refer to Register Description section. 

25 2). CPU_Rd_FIML {input: Adr; output: Rd_value} : 

26 This command will read the content of the FIML table at location Adr. Refer to Register 

27 Description section. 
28 

29 Internal Memory List (IML): 

30 The IMBID value is the address of the IML block. The content of each entry 

3 1 contains 3 bits: the valid bit, bypass bit, and readback bit. 
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1 ■ Valid: this bit indicates that at IMBID location in the Internal Memory, a 

2 cell is valid. 

3 ■ Bypass: IMBID points to a cell that is whether a bypass or non-bypass 

4 type. A bypass type cell will not go into the external SDRAM, it is ready 

5 inside the IM to go out to the Reassembly. A non-bypass type cell is in 

6 transition to go into the external SDRAM. 

7 ■ Readback: indicates that this cell is from the SDRAM and ready to go out 

8 to the Reassembly block. 
9 

10 CAM: 

1 1 CAM contains the values of BIDs (external SDRAM pointers) that are inside the 

1 2 Internal Memory. If a CAM hit occurs, an IMBID value will be returned by CAM. After 

1 3 receiving the IMBID, the MIM will go to the IML and read the content at IMBID. This 

14 content contains IMBID (Internal Memory address) and its status. If valid bit is equal to 

15 1 , it means that IMBID is valid and, therefore, this CAM hit is valid. Otherwise, a "miss" 

16 is returned. 

1 7 There are 4 main internal commands (between blocks and not CPU commands) 

18 can be issued to the MIM block (from other blocks): 

19 1). TakeJD <r IMBID: 

20 In this command, the MIM will access the FIML and take a free IMBID from it. The 

2 1 output of the command will be a free IMBID that represents a free cell inside the Internal 

22 Memory. The TakePtr will be incremented by 1 after it releases a free cell. 

23 2). RetumJD {input: IMBID} : 

24 In this command, the MIM will receive an IMBID as an input. It will return this pointer 

25 to the FIML. The ReturnPtr will be incremented by 1 after an IMBID is returned. 

26 3). Compare_CAM {input: BID; output: IMBID, status, hit/miss} : 

27 In this command, input argument is BID and output is IMBID, status (valid bit, bypass 

28 bit, busy bit) and hit/miss bit. The MIM receives BID and compares it to the entries in the 

29 table. If a match is found, the CAM will output the corresponding IMBID. The MIM goes 

30 to the IML and read its content at the location IMBID. The MIM also checks to see if the 

3 1 "valid" bit is equal to 1 at this location. If it is equal to 0 then a miss has occurred, 
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1 otherwise, it is a CAM hit. The content of IML at IMBED will include status, and a 

2 hit/miss bit. 

3 4). Wr_C AM {input: IMBED, BID, status} : 

4 This command will modify the content of the CAM table and the content of the IML 

5 table with the new values of BID and status, respectively. 

6 5). Rd_CAM {input: IMBID; output: BED, status} 

7 This command reads the contents of the CAM table and the EVIL table. 
8 

9 The CPU interface should be able to write and read to/from the CAM table: 

10 1). CPU_Wr_CAM {input: EMBED, BED, status} : 

1 1 This command allows the CPU interface to write to the CAM table and the IML table 

1 2 with new values of BID, and status, respectively. 

, n 1 3 2). CPU_Rd_C AM {input: IMBID; output: BID, status} : 

N) 14 This command will read the content of CAM table (i.e. BED) and the content of EVIL 

y3 15 table (i.e. status) 



S 16 

3 1 7 Internal Write Manager Block 



q 1 8 This block receives input from the Segmentation block and the SDRAM Read 

« 1 9 Manager, and manages the writing process into the Internal Memory (SRAM) block. 
M= 20 There are 4 main functions that this block handles independently: 

21 

22 Function 1 : (Segmentation Internal Memory). 

23 1) Segmentation asserts Seg_Mem_Available signal to the MM block to let it know that 

24 there is data available in its fifo. 

25 2) MM block sends Mem_Seg_Cntl_Pop signal to pop the cell control information of a 

26 cell. 

27 3) Cell control information is sent to the MM block. 

28 4) At a fixed delay from the time that IWM gets the cell control information from the 

29 Segmentation block, the Segmentation block starts to send cell data to the MM block 
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1 at 128-bit word at 10ns period (every other cycle of 200 Mhz clock). The 128-bit data 

2 word will be first latched into flip-flops inside the IWM. 

3 5) From FIML, get free IMBED: issue TakeED command to MIM block. 

4 6) Write data into the Internal Memory: WrJM {EMBED; Seg_Data} . 

5 7) Update CAM with new BED: Wr_CAM {IMBED; BED, valid=l , bypass=0/l , 

6 readback=0}. 

7 8) If (not "bypass"), 

8 a. Then put new BED/EMBED into end of SDRAM Write Request Queues: 

9 Wr_SWRQ {end of queue; BED/EMBED} . 



10 

1 1 Note: This block has a small FEFO to receive data inputs from the Segmentation block. 

1 2 The size of the FEFO is 4 64-byte cells with 1 28 bit wide. Data will be stored in this IWM 
O 13 FEFO before it is sent to the Internal Memory. 

S3 14 

s 15 Function 2: (Segmentation-^ Per Flow Queue). 

W 16 1) IWM sends the cell control information to the PFQ block with a command specified 



U 17 in Mem_PFQ_Com signals (i.e. "00": invalid; "01": enqueuing; "10": discard; 

L 1 8 "ll":release (for dequeuing)). 

P 19 2) PFQ receives the information, processes it and then 

H 20 a. If command = "01", then it sends back en-queue BED (Buffer ED) to the MM 

O 21 block 8 cycles later. This BED will be used as an address to write data into the 

*"~ 22 external SDRAM. If BED = "7FFFFF", then that particular cell will be 

23 dropped and not written into the SDRAM. 

24 b. If command = "10", then it sends BID = "7FFFFF" back to the MM block. In 

25 this case, data will not be written into the external SDRAM. 



26 

27 Function 3 : (Control data path to Internal Memory) . 

28 Inputs and outputs of the Internal Memory are time shared as follows (to handle dual-port 

29 128-bit wide SRAM at 200 Mhz): 

30 1) Slot 0 (period = 5ns): input from Segmentation. 

3 1 2) Slot 1 (period = 5ns): input from SDRAM Read Manager. 
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1 

2 Function 4: Microprocessor (CPU) Access. 

3 When switched to this mode, the microprocessor can write to the Internal 

4 Memory (IM) with a specified address. During this mode, there is only the CPU interface 

5 block that can access (read/write) to the IM block. Refer to the Register Description 

6 section.. 

7 CPU_Wr_IM {input: Adr, Wr_value} : write to IM at the Adr location with the 

8 value of Wr_value. 

9 

10 SDRAM Read Manager Block 

5 1 1 This block receives read requests from the SDRAM Read Request Queues and 

%y 

^ 1 2 transfers those requests with data to the SDRAM Manager. It receives read-back data 

yf 1 3 from the SDRAM Manager and sends it to the Internal Memory. 

S 14 

fey? 

N= 15 Function: (SDRAM Internal Memory). 

« 16 n SDRAM Read Manager sends data with BID and IMBED to the Internal Write 

2 1 7 Manager: Rd SRRQ {top of queue} <= BID, IMBID, SDRAM Data. 

P 18 2) Request the Internal Write Manager block to write data into Internal Memory: 

ft 19 Wr IM {IMBID; SDRAM_data}. 

20 3) Update CAM with new BID (send a command to the MM): Wr_CAM {IMBED; BID, 

21 valid=l, bypass=l, readback=l}. 
22 

23 Internal Read Manager Block 

24 This block receives input from the Internal Memory and manages its output to the 

25 Reassembly block and the SDRAM Write Manager. 
26 

27 Function 1 : (Internal Memory * Reassembly). 
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1 1) Look into CAM to see if BED from top of De-queue Queue matches with BID in 

2 ' CAM. Issue a command to MIM: Compare_CAM {BID; IMBID, status, hit/miss} . If 

3 hit/miss = 1 , get IMBID as output from CAM. Put IMBID into the output queue 

4 inside the IRM block. Assert Mem_Ras_Available to the Reassembly block. 

5 2) The Reassembly block asserts the Ras_Mem_Cntl_Pop signal to the MM block. The 

6 IRM sends the cell control information at the top of the queue by popping from top of 

7 the De-queue Queue block. 

8 3) At a specified time slot the Reassembly block asserts the Ras_Mem_Data_Pop signal 

9 to the MM block. The IRM starts to read data from the IM, RdJM {IMBID} <= 

10 ReassemblyData. 

1 1 4) Send data to the Reassembly block. Pop IMBID from the output queue. 

12 5) If(REL=l) or (REL=0 and "bypass" = 1 and "read back"=D 

Q 13 Th en return IMBID to Free Internal Memory List: Return IDflMBID}. 

H 14 6) Update CAM: Wr_C AM {IMBID; BID= 00, valid=0,bypass=0,readback=0}. 

J 15 7) If (REL = 1) then 

W 16 Release BID to FBL in PFO: assert Mem PFO Val id hieh. Mem PFO Com = 

M= 17 "11" (release BID) and Mem PFO Param = BID. 

q 1 8 (Refer to Multicast and BID/IMBED Release section) 
y= 19 

Z ;S 

P 20 Function 2: (Control data path to Reassembly and SDRAM). 

H 2 1 Inputs and outputs of the Internal Memory are time shared as follows (to handle a dual- 

22 port 1 28-bit wide SRAM at 200 Mhz): 
23 

24 1. Slot 1 (period = 5ns): output to Reassembly. 

25 2. Slot 2 (period = 5ns): output to SDRAM Write Manager. 
26 

27 Function 3: Microprocessor Access. 

28 When switched to this mode, the microprocessor can read from the Internal Memory (IM) 

29 at a specified address. During this mode, there is only the CPU interface block that can 

30 access (read/write) to the IM block. 
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1 CPU_Rd_IM {input: Adr; output: Rd_value} : read from IM at the Adr location with the 

2 value of Rd value. 
3 

4 SDRAM Write Manager Block 

5 This block receives write requests from the SRAM Write Request Queues and 

6 then gets data from the Internal Memory to send to the SDRAM Manager block. 
7 

8 Operation: (Internal Memory SDRAM). 

9 1 ) Get BID and IMBID from the top of SDRAM Write Request Queues (refer to 

10 SDRAM read/write strategy section for more details on algorithm to select proper 

1 1 banks of memory to read and to write). 

i 12 2) Look into MTM to check status of that BED: RdCAM {IMBID; BID, status}. 

Si 13 3) If"bypass"=l, 

S 14 Then do not write data to the SDRAM. Return BID to FBL. Go to last 

m 15 step (5). 

J" 1 6 Else if "bypass" =0, then get data from the Internal Memory: Rd IM {IMBID} <= 

O 1 7 IM_Data. Send IM_Data to the SDRAM Manager along with BID and IMBID. 

O 18 4) Return IMBID to the Free Internal Memory List and Update CAM: 

JJ 19 ReturnJD {IMBID} 

H 20 Wr CAM {IMBID; BID=00, valid=0, bypass=0,readback=0} . 

21 5) Pop the BID and IMBID from the top of SDRAM Write Request Queues. 
22 

23 De-queue Request Manager Block 

24 This block receives De-queuing requests from the Per Flow Queue block with 

25 other related information. It will look into the CAM to see if there is a CAM hit. If it is, it 

26 will modify CAM to reflect its new status, that is now a "bypass" data and should not be 

27 written into the SDRAM. It then lets the Internal Read Manager take data out of the 
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1 Internal Memory and send it to the Reassembly block. If there is no CAM hit, it will send 

2 a read request to the SDRAM Read Request Queues to get data out from the SDRAM. 
3 

4 Function: (Per Flow Queue Internal Memory, SDRAM). 

5 a. Store BID, PortID and other cell control related information for De- 

6 queuing process into the end of De-queue Queue. 

7 b. Look into CAM to see if BED is matched with BID in the CAM. 

8 CompareCAM {BID; EMBED, status, hit/miss} 

9 c. If BID is found in CAM (hit/miss = 1 ) and "readback" = 0 (not outgoing 

10 cell) 

11 lf"bypass" = 0 

12 Then get IMBED. Modify CAM: Wr CAM {IMBED; BID, valid=l, 
y 13 bypass=l, readback=0}. 

H 14 Else if no CAM hit, 

j 1 5 Then get free EMBID from Free Internal Memory List: Take DD. Write 

W 16 BED and EMBID into end of SDRAM Read Request Queue. 

M= 17 

jjj 18 Multicast and BID/IMBID Release 

« 1 9 During Dequeuing the PFQ sends to the MM block a command to indicate 

P 20 whether to release the Dequeue BID to the Free Buffer List (in the PFQ) or not to release 

2 1 the Dequeue BID due to Multicast. Depending on the type of data inside the Internal 

22 Memory, the IMBED is also released to the Free Internal Buffer List (FD3L) in the MTM 

23 block. Figures 1 76- 1 78 depicts the mechanism of the release command: 



24 

25 SDRAM Manager (SM) 

26 See Figure 179. The DDR SDRAM Controller (SM) receives the memory write 

27 (enqueue) and read (dequeue) requests from the Internal Memory Manager (EMM). All 

28 operations are done on a per controller basis independent of the other controller. Each of 

29 the two SDRAM Manager blocks, SMO and SMI consists of: 
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1 • RD ID FIFO 

2 • WR ID FIFO 

3 • BANDWIDTH OPTIMIZER BLOCK 

4 • REQUEST SERVICE BLOCK 

5 • CONTROLLER INTERFACE BLOCK 

6 • BUFID ADDR DECODE BLOCK 

7 • DENALI DDR_SDRAM MEMORY CONTROLLER BLOCK 

8 SMO and SMI blocks each control corresponding pair of SDRAMs. It uses 128-bit 

9 data bus at 200 MHz in order to support 2X OC-1 92 transfer rate (20 Gbps). Each 

1 0 DDR_SDRAM Controller has the capability of peak data transfers at 1 6 bytes (or 1 28 

1 1 bits) per 5ns or 25 .6 Gbps for a total memory bandwidth of 5 1 .2 Gbps in the 
«. 12 DDR_SDRAM Manager (SM). Therefore, the DDR_SDRAM Manager must 

€1 13 simultaneously process two memory accesses, one each to SMO and SMI in order to 

14 sustain the required bandwidth. 

1 5 The BANDWIDTH OPTIMIZER block decides which of 8 possible operations, 
3 16 4 reads from each bank and 4 writes to each bank to service for optimal data throughput 

1 7 from the SDRAMs. It looks at the current state which is the last processed request and 

O 1 8 the outstanding requests in the per bank queues and picks one based on an algorithm 

5 19 detailed below to optimize the data throughput from/to the SDRAMs. Some of the 

E 20 avoided situations include sequential same bank operations with their bank activate and 

21 precharge delays, read followed by write commands with the associated pitfall of turn 

22 around cycle delays, etc. 

23 The Bandwidth Optimizer Block will scan through the 8 queues, 4 reads and 4 

24 writes per Memory Manager, DM0 and DM1 , and select an operation request to the 

25 appropriate bank based on the SDRAM Read/Write Strategy discussed below. 

26 • Each series of Read requests to external memory will have to follow by a Write 

27 request to a different bank of memory from the accessed bank of the last Read. 

28 • Within a series of Read requests, consecutive Reads should have different banks. 

29 • Within a series of Write requests, consecutive Writes should have different banks. 

30 For Example: 
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1 Rd(bO) ->Rd(bl) ->Rd(b2) ->Rd(b3) ->Wr(bO) ->Wr(bl) ->Wr(b3) ... 

2 At the same time the read and write request BID/TMBID are stored in the READ 

3 ID FIFO and WRITE ID FIFO respectively to be returned to the IMM with the read 

4 data or a write request confirmation respectively. For write request, IMBID is returned so 

5 that IMM can update the CAM and release IMBID to Free Internal Memory List. For 

6 read request, BID/IMBID are returned so that IMM can store the read data to IM and 

7 release BID to Free Buffer List. 



8 The REQUEST SERVICE BLOCK is activated with the particular request to be 

9 serviced by the Bandwidth Optimizer block. It routes the data back and forth 

10 appropriately to and from the dram and communicates with the Address Decode block 

1 1 and the Memory Controller Interface block. It also handles the back pressure from the 
0 12 Denali Memory Controller in case the command queues or the data queues in that block 
SI 13 have reached their limit. 

« 14 The BUFFERID ADDRESS DECODE BLOCK does the address translation of 

W 1 5 the BID to the absolute byte address and feeds this to the Memory Controller Interface 

u 16 block via the Request Service Block. 

17 The CONTROLLER INTERFACE BLOCK formats all the commands for the 



p 1 8 Denali Memory Controller. It processes the address and data part and sends them to the 

rj 19 Memory Controller along with required control signals in the format expected by the 

Q 20 Memory Controller. It also receives the data from the memory controller for read 

21 operations and sends it to the Request Service Block. It will pass on control signals from 

22 the Controller to the Request Service block signaling that the controller command or data 

23 queue is full and when it is ready to receive the next request 

24 From the above block onwards the interface of the DENALI MEMORY 

25 CONTROLLER and the SDRAMs is detailed in the diagram below. The Memory 

26 Controller takes in the Address and Data from the Memory Controller Interface Block 

27 and formats them into appropriate RAS and CAS commands for the DRAM memories. It 

28 also does the refresh of the SDRAM memories. 

29 See Figure 180. 
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1 CPU Interface 

2 Assumption: CPU access requests are infrequent and are for debugging and testing 

3 purpose only. 

4 SM services the CPU request along with the en-queuing and de-queuing requests 

5 with equal priority. Whenever CPU requests an access to DDR-SDRAMs, SM will 

6 process the request at the end of one round scanning and servicing the 8 queues of the 

7 read/write requests. In order not to interrupt the bandwidth for enqueuing and dequeuing, 

8 each CPU access will be serviced after X number of rounds of servicing the read and 

9 write queues. (The X number is programmable through a register) 

10 Since the en-queuing cells and de-queuing cells are 64-byte and hence the DDR 

^ 1 1 SDRAMs are programmed to have burst length of 4 (64x8 = 4x128), the CPU data aceess 

m 12 should be in the same cell unit, 64-byte. 
*J 13 

14 Memory Description 

OS 15 

16 Summary of Memories: 

g 17 See Figure 181. 

5 18 

jj 19 Summary of CAM: 
H- 20 See Figure 182. 

21 Interface 

22 

23 Segmentation Engine 

24 See Figure 183. 
25 

26 Per Flow Queue Engine 

27 See Figure 184. 
28 

29 DDR-SDRAM 

30 See Figure 185. 
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1 

2 Reassembly 

3 See Figure 186. 
4 

5 CPU Interface 

6 See Figure 187. 
7 

8 Global Sync 

9 See Figure 188. 
10 

1 1 Interface with DDR-SDRAM 

Q 1 2 The Memory Manager can support up to 5 1 2 Mbytes of external memories using 

^ 13 DDR-SDRAM. The DDR-SDRAM device that is available at this time is the 1M x 32bit 

"Si 

Nl 14 x 4banks (128 Mbits). The memory size can be programmed using register 

hj 15 Mem_Config[2:0] (refer to the Register section). 

® 1 6 2 DDR-SDRAM devices are required to support a data bus of 64bit. Hence 2 data 

3 17 strobes, Mem_SDR_DQS_Lo and Mem_SDR_DQS_Hi, are used to for the lower and 

p 18 upper 32bit of the data bus respectively. 

Q 19 I A duplication set (set 1) of memory address and control signals (including 

□ 20 memory clock) is available for each controller in order to improve its signal driving 

^ 2 1 capability when the number of devices are increasing (up to 1 6 devices per controller). 

22 Hence each interfacing signals will not drive more than 8 devices in any configuration of 

23 memory sizes. 

24 This second set of address and control pins can be activated by programming the 

25 register Mem_Config[3]. 
26 

27 Example 1: 

28 Refer to Figure 1 89 for the configuration of 64 Mbytes of memory. 

29 Mem_Config[3] = 0. 



30 

31 Example 2: 
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1 Refer to Figure 1 90 for the configuration of 256 Mbytes of memory. 

2 Mem_Config[3] = 1. 
3 

4 Register Description 

5 See Figure 191. 
6 

7 Commands 

8 CPU_\Vr_SDRAM 

9 See Figure 192. 



CPU Rd SDRAM 

See Figure 193. 
CPU_Wr_ FIML 

See Figurel94. 
CPU_Rd_FIML 

See Figure 195. 

16 CPUJWr CAM 

17 See Figure 196. 

18 CPU Rd CAM 

19 See Figure 197. 

20 CPU_Init_MM 

21 See Figure 198. 



10 

3_ -ir ;: 

8 12 

^ 13 
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1 
2 
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11 
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Principle of Operation 

Initialization Setup 

At power-on-reset, the entire block is reset by MEM_Reset_L signal. After 
power-on-reset, all registers are at default values. 

CPU can issue the CPU_Init_MM command (refer to the "Register section") that 
will go through proper steps to initialize all internal memories inside the Memory 
Manager block. 

Operational Flows 
Normal Mode: 

The normal mode is the default mode. The Memory Manager handles both the 
enqueuing and the dequeuing processes. 

Enqueue: 

Segmentation to MM: 

1. Segmentation block has data available for MM block, it starts to assert 
Seg_Mem_Available signal. 

2. MM block asserts Mem_Seg_Cntl_Pop signal to get cell control information from 
Segmentation block. 

3. At a specified time slot the Segmentation block sends cell control information to the 
MM block. 

4. The MM block stores cell control information in the IWM block. 

5. At a specified time slot the Segmentation block starts to send data to the MM block 
through the Data bus (128 bits) in 4 clock cycles at 10 ns period (every other clock 
tick of the 200 Mhz clock). 
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1 6. The IWM block stores data inside the IWM FIFO (4 64-byte cells). 

2 7. After receiving the free BID from PFQ (see below), the IWM block write appropriate 

3 data from the IWM FIFO into the Internal Memory (BVI). 
4 

5 

6 MM to PFQ: 

7 The IWM block sends cell control information to the PFQ block on the 

8 Mem_PFQ_Param along with a command via the Mem_PFQ_Com bus. The 

9 Mem_PFQ_Valid signal is asserted high to let the PFQ block knows that the data bus and 

10 command bus contains valid data. If the Discard signal is not asserted then the 

1 1 Mem_PFQ_Com value should be 01 (enqueue). . If the Discard signal is asserted then the 

1 2 Mem_PFQ_Com value should be 1 0 (discard) . 

=f 13 1 . If it is an enqueue command, the PFQ block sends back to the MM block a BID at a 
SJ 14 specified number of clocks later via the PFQ_Mem_Param bus.This BID is used as an 

. r 15 address to write data into the external SDRAM. The PFQ_Mem_Valid signal is 
W 16 asserted and the PFQ_Mem_Com value is 0 1 . 

ii 17 2. IWM block receives BID and tags it along with the associated data being stored in the 
q 18 IWM FIFO. 

jjP 19 3. If it is a discard command, the PFQ block sends back a BID with value of "7FFFFF" 
5 20 to the MM block at that specified time slot. In this case, data from the Write FIFO 
F? 2 1 will not be written inside the external SDRAM (it will be flushed out of the FIFO). 

22 4. If the PFQ decides to drop the cell (even when the command is not a discard), it will 

23 send to the MM block a BID with value of "7FFFFF". In this case, data from the 

24 Write FIFO will not be written inside the external SDRAM (it will be flushed out of 

25 the FIFO). 
26 

27 MM to SDRAM: 

28 

29 1 . Data from the Segmentation is temporarily stored in the IWM FIFO. 
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1 2. After receiving a free and valid BID from the PFQ, the IWM block requests a free 

2 IMBED from the Memory ID Manager (MIM) block. The CAM is also updated 

3 properly with the new 1MB ID and BID. 

4 3. IWM writes data from the IWM FIFO to the Internal Memory (Dvl) block at address 

5 1MB ID. 

6 4. If the cell is not a "bypass" cell, then IWM sends BID and IMBID to the SDRAM 

7 Write Request Queue (SWRQ) block. Place the information at the bottom of the 

8 queue that associates with memory bank of the BID address. 

9 5. The SDRAM Write Manager (SWM) block goes through the "Write Bank Selection 

1 0 Algorithm" and it selects a BID/IMBID pair from a particular queue inside the 

1 1 SWRQ block. The SWRQ sends that IMBID to the MIM block for status verification. 

12 6. If that cell is marked as a "bypass" cell (this means that it is being dequeued and 

5 1 3 ready to go out to the Reassembly block), then do not write data to the SDRAM. Pop 

M 14 the BID/IMBID from the top of the queue inside the SWRQ block. 

J 15 7. If that cell is not valid as indicated in the status (this means that it has gone out to the 

S 1 6 Reassembly block), then do not write data to the SDRAM. Pop the BID/IMBID from 

yy 

M 1 7 the top of the queue inside the SWRQ block. 

q 18 8. If that cell is marked as a "non-bypass" cell, then 

2 1 9 a). The SWM block sends request to the IRM block with BID/IMBID. 

U 20 b). IRM reads data from the EvI block (RdEvl {IMBID}) and send this data to the 

H 2 1 SDRAM Manager block along with BID/IMBID. 

22 c). Return IMBID to the MIM block and update its CAM: 

23 - ReturnID {IMBID} 

24 - Wr_C AM {IMBED; BDD=00, valid=0, bypass=0, readback=0} 

25 - Pop BID/IMBID from the top of the queue inside the SWRQ block 

26 Dequeue: 
27 

28 PFQ to MM: 

29 1 . At a specified time slot, PFQ sends to the Read Request Queue & Manager (RRQM) 

30 block dequeued cell control information that includes a BED via the Data2 bus. Both 
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1 Valid2 and Deq signals are asserted to let the MM block knows that the data bus is 

2 valid and contains dequeue information. 

3 2. The RRQM block stores cell control information at the end of the queue. 

4 3 . The RRQM block sends a BED from the top of the queue with a read request to the 

5 Memory Read/Write Controller (MRWC). 
6 

7 SRAM to MM: 

8 1 . The MRWC block monitors the Read FIFO. If it is not full, the MRWC sends a 

9 dequeue BID to the SRAM Interface (SI) block with a read command ("10"). 

10 2. The SI block puts the BED and the command in a queue. 

11 3. The SI block executes a read command to the external SRAM with the BED as an 

12 address. Data is read out from the external SSRAM and sends to the Read FIFO. The 
3 1 3 associated BED will be placed inside the Read FIFO along with the data. 

H 1 4 4. When a complete cell is read back from the SSRAM, the Read FIFO sends its status 

.2 1 5 and the associated BID to the MRWC block. 

W 16 5. The MRWC block sends the BID back to the PFQ block via the MM_PFQ_Data bus. 

P 17 The Valid is asserted and the Com value is "1 1" (dequeue). 

L 1 8 6. PFQ receives the BED and releases it back to the Free Buffer List block. 

U:I 19 

ST* 

il 20 MM to Reassembly: 

21 1 . The Read Request Queue & Manager (RRQM) block monitors the Read FIFO status. 

22 2. Whenever there is a complete cell available inside the Read FIFO, the RRQM sends 

23 the MM_Ras_Available to the Reassembly block. 

24 3 . The Reassembly block asserts the Cntl Pop signal to the MM block. The RRQM 

25 block sends the cell control information at the top of the queue to the Reassembly 

26 block at a specified time slot. 

27 4. At a specified time slot the Reassembly block asserts the Data_Pop signal to the MM 

28 block. The Read FEFO starts to send data out to the Reassembly block at a specified 

29 time slot via the 128-bit Data bus at 10 ns period (every other clock tick of the 200 

30 Mhz clock). 
31 



B 



107 



AZA-001/2001-P00 




PATENT 



1 Flow charts for Functional Operations 

2 See Figures 199-205. 
3 

4 Reassembly 

5 The Reassembly Engine is a state machine that reassembles the stored switch cells 

6 into outgoing ATM, AAL-5, packets, or switch cells. 

7 Main Features 

8 - Can provide a maximum of 1M total flows. 

9 - Uses PER PORT queues for re-assembly. 

10 - Reassembles segmented cells back into IP packets. These packets were 

1 1 segmented into cells for switching at the ingress side and need to be 

12 reassembled back 

1 3 - Maximum packet size of 64 KB can be handled 

14 - AAL5 trailer field check for length and CRC-32 for all flows that have to be 

1 5 re-assembled and sent out as packets. 

16 - L2 CRC check for packets that have the L2 CRC check enabled. 

1 7 - Can strip up to 1 6 bytes of Header data. 

1 8 - Reassembles 48 byte AAL5 cells or 52 byte ATM cells, both stored as 64 byte 

19 memory cells into outgoing packets, ATM cells or switch cells. 

20 - No lookup, pass-through traffic 

21 - For ATM 

22 o VPI, VCI header translation 

23 o Operation, Administration and Maintenance (OAM), Cell Loss Priority 

24 (CLP) and Explicit Forward Congestion Indication (EFCI) replacement 

25 o Mark End of Packet in Payload Type Indicator (PTI) Field for AAL5 

26 Egress Cells 

27 o Optional MPLS tag append 

28 o Encapsulate raw cells into packets for routing through packet fabrics. 

29 o Encapsulate ATM cells into packets for routing through packet fabric. 

30 - MPLS lookup 

31 o POS (PPP/TP) packet format 
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o MPLS push/pop 

o MPLS header translation 

o Decrement TTL and drop if TTL is one before decrementing 

- It has one re-assembly going on for each of the ports that are conFigured and 
active, supports a maximum of 64 ports plus a CPU port. 

- The output of this block goes to the PHY PORT. 
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1 - Supports the traffic represented in Figure 206. 

2 - Applications for various traffic types 

3 - Types of reassembly: 

4 o TYPE 1 : Ingress TM (ATM, AAL5, Packets) to 64B Switch Cells 

5 o TYPE 2: Ingress ATM cells to Packet Data 

6 o TYPE 3: Ingress AAL5 cells to Packet Data 

7 o TYPE 4: Ingress Packets to Packet Data 

8 o TYPE 5: Egress (ATM) to ATM cells with Translation 

9 o TYPE 6: Egress (AAL5) to Packet Data 

10 o TYPE 7: Egress (Packets) to AAL5 Cells 

11 o TYPE 8: Egress (Packets) to Packet Data 
12 

1 3 - Calculates and checks the CRC-32 for AAL-5 reassembly packets 

14 - Each port has a 32-bit counter to count packets/cells transmitting from that 

15 port 

1 6 - Each port has a 48-bit counter to count number of data bytes transmitted 

17 through that port 

18 - Each port has a 16-bit counter to count packet CRC errors. 

1 9 - Each port has a 1 6-bit counter to count TTL Timeout errors. 

20 - Internal memory to store control and data for the 64 ports 

21 o Dual-port SRAM 

22 ■ CPU Data - 32 x 64 bits to store CPU data 

23 ■ Data - 1024 x 64 bits to store Cell data 

24 ■ Header - 256 x 64 bits to store Header data 

25 ■ Output Port Control - 128 x 21 bits to store Output Port Control 

26 information and Next Pointer links. 

27 ■ Output Port Queue - 64 x 24 bits to store Output Port Queue 

28 Pointers to the Data cells 

29 ■ CRC - 64 x 32 bits to store the partial CRC for each port 

30 o Single-port SRAM 



110 



AZA-00 1/2001 -POO' 



PATENT 



1 ■ Reassembly Table - 64 x 1 8 bits to store packet length, SOP and 

2 EOP 

3 ■ Free Buffer -128x10 bits to maintain the free buffer list for data 

4 SRAM 

5 ■ Statistic - 64 x 96 bits to store packets, data and errors received for 

6 each port 

7 - Operates at 200 MHz 
8 

9 Functional Description 

10 See Figure 207. The Reassembly Engine reassembles the outgoing data from 

1 1 cells stored internally into ATM cells, Switch cells, or Packet data based on the 

12 application. 

\0 13 As shown in Figure 206 there could be 8 different types of reassembly. The 

S3 

S| 1 4 operation of the Reassembly Engine for each of these types is explained in detail in the 

f*i 15 following sections. 

1 16 

L 17 TYPE 1 Reassembly - Ingress Memory cells => Switch cells 

2 1 8 Type 1 reassembly maps the 64-byte cells in Memory into switch cells for 

Q 19 transmission by adding a Switch Header. This Reassembly type handles Application 

q 20 types 0-3 & 7. Figure 2 shows the one to one mapping of ATM Memory cells into switch 

!fA 2 1 cells. The switch header will be 8 or 1 6 bytes long with the required cell fabric header 

22 being first and our 4-byte header following with the remaining bytes being Pad. Type 1 

23 Reassembly also informs the SPI interface of the number of valid bytes in each 8-byte 

24 word and which Cell is the SOP and which Cell is the EOP for Packet traffic. For AAL5 

25 cells the cells were was stored in memory as AAL5 switch cells with the AAL5 trailer at 

26 the end and Pad to fill the cell to 64 Bytes. For Packet traffic the cells were stored in 

27 memory as switch cells with an AAL5 like trailer added to pass on the packet length 

28 information. Figure 3 shows the AAL5 encapsulated cells and Figure 4 shows the pseudo 

29 AAL5 encapsulated packets. For Ingress TM types 0 & 1 that transfer ATM cells the 

30 OAM, EFCI and CLP bits will be inserted in our 4 byte switch header but for Ingress TM 

3 1 packets, types 2 & 3, the bits will be set to zero. 
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See Figures 208-10. 

TYPE 2 Reassembly - 52 Byte ATM cells => Packet Data 

Type 2 Reassembly maps the 52-byte ATM cells stored as 64 byte cells in 
Memory into packets and adds a packet header before sending it on to the SPI interface. 
Only one ATM cell will be mapped into each packet. Eight of the 12 bytes of Pad data at 
the end of each cell will be removed before the packet is sent out. This will allow us to 
send out a 64-byte packet including 8-bytes of header. If the Switch Packet Header is 16- 
bytes then the packet will be 72-bytes long. The OAM, EFCI and CLP bits will be 
passed in our 4 bytes of the Switch Packet Header. 

See Figure 211. 
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TYPE 3 Reassembly - 48 Byte ATM cells => Packet Data 

Type 3 Reassembly maps the 48-byte ATM cells stored in Memory as 64 byte 
cells into packets by stripping the 16 bytes of PAD from each cell and then optionally 
stripping the L2 Packet Header and adding a Switch Packet Header before sending it on 
to the SPI interface. It must check the L2 CRC to ensure the packets integrity before 
sending on to the switch fabric as it also checks the CRC in the AAL5 Trailer and 
removes the AAL5 encapsulation. The OAM, EFCI and CLP bits will be passed in our 4 
bytes of the Switch Packet Header. It must check the L2 CRC to ensure the packets 
integrity before sending it to the switch fabric as the L2 CRC being sent out is now 
incorrect due to the removal of the L2 Header and must be replaced by the Egress chip 
when it adds a new L2 Packet Header. The Lookup engine will replace the L2 Header 
and the Segmentation will calculate the new CRC. 

See Figure 212. 

TYPE 4 Reassembly - Switch cells => Packet Data 

Type 4 Reassembly maps the AAL5 like Switch cells stored in memory into 
packets removes the AAL5 encapsulation and may add an 8 or 16-byte Switch Packet 
header. This can also operate without a lookup operation. (Traffic Shaper, Packet in, 
Packet out) 

See Figure 213. 

TYPE 5 Reassembly - 52 Byte ATM cells + PAD => ATM cells with Translation 

Type 5 Reassembly translates the ATM Header by replacing the 28-bit VPI/VCI 
then replaces the OAM, CLP & EFCI bits before the cell is sent out. The extra 12 Bytes 
of Pad data from the end of each ATM cell stored as 64-byte cells in Memory is stripped 
before sending it out as a 52-byte ATM cell. Using the data byte enables of the SPI 
interface strips the extra Pad. 

See Figure 214. 
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1 TYPE 6 Reassembly - 48 Byte ATM cells => Packet Data 

2 Type 6 Reassembly maps the 48-byte AAL5 encapsulated ATM cells stored in 

3 Memory as 64-byte cells into packets by stripping the 16 bytes of PAD from each cell 

4 and then performing a MPLS operation based on the contents of the Flow ED look-up 

5 table before sending it on to the SPI interface. It also checks the CRC in the AAL5 

6 Trailer before removing the AAL5 encapsulation. If the MPLS TTL is one before being 

7 decremented the packet will be dropped in Reassembly. 



8 See Figure 215. 

9 

1 0 TYPE 7 Reassembly - 48 Byte ATM cells => AAL5 cells 

1 1 Type 7 reassembly adds the 4 byte ATM Header along with 4 bytes of PAD, so it 



12 is 8 byte word aligned, to the 48-byte AAL5 encapsulated ATM cells stored in Memory 

O 13 as 64-byte cells, strips the 16 bytes of Pad, replaces the OAM, CLP & EFCI bits in the 

S3 14 ATM Header and then transmits them as AAL-5 cells. The 4 bytes of alignment PAD 

^ 15 will be stripped using the byte enables when transmitting to the SPI. The EOP AAL-5 

UJ 1 6 cell is marked using the PTI field in the ATM Header. 

OS 

U 17 See Figure 216. 

U 18 

m l9 XYPE 8 Reassembly - Switch cells => Packet Data 

rT 20 Type 8 Reassembly maps the AAL5 like Switch cells stored in memory into 

H 21 packets and performs a MPLS operation based on the contents of the Flow ID look-up 

22 table. It also checks the CRC in the AAL5 Trailer before removing the AAL5 

23 encapsulation. If the MPLS TTL is one before being decremented the packet will be 

24 dropped in Reassembly. 

25 See Figure 217. 
26 

27 Block Descriptions 

28 See Figure 218. 
29 
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1 Description of INPUT/ENQUEUE_SM Block 

2 The CTRLSM block receives PID, FID, TYPE, EOP, SOP, CLP, EFCI and 

3 MEMRASAV AIL ABLE from the Memory Manager. When AVAILABLE is asserted 

4 the CTRL SM pops the control information from the Memory Manager control FIFO. 

5 The state machine then reads from the control SRAM using PID as the address to retrieve 

6 parameters for that port. The control state machine also requests a pointer to a free buffer 

7 from the Free Buffer List (FBL). Then it loads the parameters into the internal registers 

8 and starts the D ATA SM. The type of Reassembly is decoded from the TYPE field by 

9 the control state machine and the enable signal for each type is passed to the data state 

10 machine. The SOP and EOP signals inform the Reassembly engine when the first and 

1 1 last cells of a packet are received. 



12 The FID is used to read the external Header Look-Up SRAM to retrieve the 

O 13 appropriate header and MPLS information for that Flow Id. This Header is passed on to 

%| 14 the DATA SM to be added to the reassembled cell. 

J 1 5 The PID and other control signals are read from the Memory Manager by the 

tbir 

U3 16 Reassembly engine 1 clock before the data to give the control state machine enough time 

5 1 7 to read the per port parameters from the SRAM and load into internal registers, retrieve a 

H, 1 8 free buffer pointer from the FBL and to start the header read from external memory. 



2 19 

U 20 Description of OUTPUT/DEQUEUE_SM Block 



O 2 1 The data state machine starts when it receives the start signal from the control 

22 state machine. Once started, the data state machine reads 64-bytes of data from the 

23 Memory Manager data FIFO. The data state machine will also determine from the 

24 TYPE field whether to check the CRC. 

25 When a complete cell is read and the data state machine has added the appropriate 

26 header information obtained from the external Look-Up SRAM, the data state machine 

27 informs the control state machine that it is finished reading the data for that cell. The 

28 control state machine then stores the per-port parameters back in the reassembly control 

29 table. The data state machine is then ready to receive more data. 

30 "Once the next cell has been received and determined not to be an EOP cell then 

3 1 the previous cell for that port is queued into the Q_FIFO for transmission to the SPI. If 
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1 the next cell is an EOP then the whole cell must be read and the length determined from 

2 the AAL5 trailer to ensure the proper amount of valid data in the previous cell before it is 

3 queued for transmission. 

4 Once the data for a cell has been stored and the control information is queued in 

5 the Q_FIFO, the Reassembly asserts READY to the SPI interface so it can come and Pop 

6 the Q_FIFO for the control information for that cell. Once the Q_FIFO has been Popped 

7 by the SPI, the Reassembly will start reading the data associated with that control 

8 information and send it on to the SPI 64 bits per 200 Mhz clock cycle until the complete 

9 cell is transmitted. A Data_Valid signal will be active when the data is valid on the bus 

10 and 8 byte enable signals will inform the SPI which bytes have valid data in each 64 bit 

1 1 transfer. This operation will allow the SPI to control the flow of data by controlling 

1 2 when the Q_FIFO is Popped and also strip the extra pad bytes when necessary. 

□ 13 The data state machine has two types of counters, one for operation and one for 

Ci 14 statistic. The cell byte count and packet length counters are for operation. The packet 

^ 1 5 received count and data received count and CRC errors are for statistic. All counters are 

UJ 16 per port basis. 
17 

s 1 8 Description of CPU I/F Block 

m 1 9 This is a standardized interface to the CPU in the system. It will be instantiated 

Q 20 within each block that requires read/write access to configuration registers or internal and 

O 21 external memory arrays. During system operation the reassembly engine will use this 

^ 22 interface for writing configuration registers and the header lookup table in external 

23 SSRAM as well as to receive status and statistic information. The CPU will also receive 

24 Data Cells or Packets through this interface using the CPU Port Buffer (port 65). 
25 

26 Description of CPU_PORT Buffer Block 

27 This is a 32 x 64 bit internal Dual Port SRAM to store the incoming data on a per 

28 memory cell basis such that the Reassembly Engine can reassemble the data into the 

29 proper cell configuration before sending it to the CPU. The memory is organized as 4 

30 64-byte buffers to allow 4 cells for the CPU port to be stored. 
31 
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1 Description of REASM_TABLE_SRAM Block 

2 This is a 64 x 1 8 bit dual-port internal SRAM to store the parameters for the 64 

3 ports. This RAM is accessed twice (1 read and 1 write) by the control state machine for 

4 each data cell that is received for a particular port. 
5 

6 Description of FBL Block 

7 See Figure 219. This is a 128 x 10 bit single-port internal SRAM to store the free 

8 pointers to the cells in the data SRAM. This list needs to be initialized by software after 

9 reset to have the free pointers. The control state machine requests a free pointer by 

1 0 reading the list, and the data state machine returns a pointer by writing back to the end of 

1 1 the list when the SPI receives a complete cell from the Reassembly block. 

O 12 

§ 13 Description of CRC_GEN Block 

H 1 4 This block generates the CRC that covers the PDU (Protocol Data Unit) in the 

S 1 5 AAL-5 format. The polynomial used is CRC-32 as shown in Figure 220. There is a 

W 1 6 minimum of 3 clocks delay in generating the CRC because the internal data path is 64 

1 7 bits and the CRC polynomial is calculated 32 bits at a time. The CRC is generated in 2 

5 1 8 stages, first the upper 32 bits and then the lower 32 bits. 

O 19 The reassembly engine calculates CRC-32 for the AAL-5 data. The CRC_GEN 

y. 

b 20 block reads the partial CRC with port ID as the address to the CRC SRAM. The 

2 1 CRCGEN block loads the partial CRC into the CRC engine and starts calculating as the 

22 data is coming in from the memory manager. When the complete cell is loaded in the 

23 internal SRAM the CRC GEN block writes back the partial CRC into the CRC SRAM, 

24 again using port ID as the address. At the end of a packet, the CRC is compared to the 

25 CRC in the AAL-5 trailer and the packet is marked as bad when sent to the SPI out 

26 interface if the compare fails. 
27 

28 Description of CRC_TBL Block 

29 This is a 64 x 32 bit dual-port internal SRAM to store the partial CRC for each 

30 port. The data state machine controls the read or write to this table based on the traffic 

31 type. 
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1 

2 Description of DAT A_PIPE Block 

3 The DATA_PIPE block delays the data to the Data SRAM so that it matches the 

4 delay to lookup and inserts the appropriate header information and generates and check 

5 the CRC-32 for all Packet reassembly. 
6 

7 Description of D AT A_SRAM Block 

8 This is a 1 024 x 64 bit internal Dual Port SRAM to store the incoming data on a 

9 per memory cell basis such that the Reassembly Engine can reassemble the data into the 

1 0 proper cell configuration before sending it to the SPI. The memory is organized as 1 28 

1 1 64-byte buffers to allow 2 cells for each port to be stored. 
12 

1 3 Description of HEADER_SRAM Block 

14 This is a 256 x 64 bit internal Dual Port SRAM to store the Header data on a per 

1 5 memory cell basis such that the Reassembly Engine can reassemble the data into the 
i 1 6 proper cell configuration and attach a programmable Header to it before sending it to the 

1 7 SPI. The memory is organized as 1 28 1 6-byte buffers to allow 2 1 6-byte Headers for 

1 8 each port to be stored. 
19 

20 Description of OUTPUT PORT QUEUE Block 

2 1 This is a 64 x 3 0 bit single-port internal SRAM to queue information of the 

22 complete cells. The data state machine queues the cell control information when a 

23 complete cell is ready to send to the SPI interface. The Reassembly Engine sends an 

24 Available signal to the SPIO when the Q_FIFO is not empty telling the SPIO that at least 

25 one cell of data is available for transmission. 

26 Control information delineates the SOP, EOP and SOB as well as the Port 

27 Number and whether the packet data had a CRC error or another error condition 

28 occurred. The FIFO also holds the 1 0-bit pointer to the data cell so the Reassembly will 

29 send the associated data with that control information. 
30 



3 
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1 Description of OUTPUT CONTROL_SRAM Block 

2 This is a 64 x 30 bit single-port internal SRAM to queue information of the 

3 complete cells. The data state machine queues the cell control information when a 

4 complete cell is ready to send to the SPI interface. The Reassembly Engine sends an 

5 Available signal to the SPIO when the Q_FIFO is not empty telling the SPIO that at least 

6 one cell of data is available for transmission. 

7 Control information delineates the SOP, EOP and SOB as well as the Port 

8 Number and whether the packet data had a CRC error or another error condition 

9 occurred. The FIFO also holds the 1 0-bit pointer to the data cell so the Reassembly will 
1 0 send the associated data with that control information. 

11 

□ 12 Description of External Look-Up Memory 

J(3 13 This is a 2M x 72 bit external SRAM to store the Header control information per 

Nl 14 FLOWED. This includes the appropriately programmed new header for the particular 

as. 

M 15 FLOW or me necessary MPLS manipulation control information for that FLOW. This 

" 16 can handle 1 M FLO Ws with 1 6 Bytes of header and 2 bytes of control information per 

i 17 FLOW. 

S 18 

O 19 Description of MPLS Label Manipulation 

p 20 The FLOW-ID Lookup table will contain the MPLS label operation instructions 

2 1 and new labels. Three bits in the first control word will contain the encoded instructions 

22 for what MPLS operation to perform. The operations will be either replace the MPLS 

23 tag, Push the MPLS tag once or twice or Pop the MPLS tag once or twice. After each 

24 MPLS operation the original MPLS TTL value will be decremented. If the TTL value is 

25 one before being decremented then the packet must be dropped. This will be done by not 

26 enqueuing the packet into the output Queue FIFO and subsequently dropping the 

27 following cells from the FLOW until the EOP is reached. 

28 If two labels are being pushed then Label 1 will be pushed on first and Label 2 

29 will be pushed on second. If one label is pushed label 1 will be used. If two labels are 

30 popped the third label will be the one remaining on the top and if one label is popped then 

3 1 the second label will be remaining on top. 
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1 The TTL will be decremented as follows: 

2 o Replace MPLS Tag - Decrement MPLS TTL 

3 o Push 1 MPLS Tag - Decrement original MPLS TTL 

4 o Push 2 MPLS Tags - Decrement original MPLS TTL 

5 o Pop 1 MPLS Tag - Decrement original MPLS TTL 

6 o Pop 2 MPLS Tags - Decrement original MPLS TTL 
7 

8 TAGS 

9 Figures 221 and 222 show the data formats for each of the supported protocols. 



10 The specifications are not hard coded. The port table will be used for the header's 

1 1 location. Each field is 8 bits wide with 12 bits VPI and 16 bits VCI field, for a total of 28 

12 bits lookup tag. This 28 bit lookup tag is located in the 1 st 64-bit word, bits 32 thru 63. 
Q 13 The PPP protocol is part of the link layer header. The tag is bit 60:41 of the first 64 bits 

%| 14 word, 
.ft 15 

US 16 ATM TYPES 

2 1 7 If the type of the traffic is ATM, then the CLP, EFCI, and OAM bits are included 

L 1 8 in the Switch Header for Ingress operation and inserted into the ATM Header for Egress 

u 

m 19 operation. See Figure 223. 
U 20 

j-f 21 Description of Memories 

22 There is one external Memory interface and 1 0 internal memory blocks for the 

23 Reassembly Engine. There is a 64 x 1 8 bit single port SRAM for a Reassembly Table 

24 SRAM, a 128 x 18 bit dual-port SRAM for Output Port Control information, a 64 x 18 bit 

25 dual-port SRAM for Output Port Queue Pointer information, a 128 x 7 bit single port 

26 SRAM for a Free Buffer list, a 1 024 x 64 bit dual-port SRAM for DATA SRAM, a 256 x 

27 64 bit dual-port SRAM for HEADER DATA, a 36 x 64 bit dual-port SRAM for CPU 

28 DATA, two 64 x 32 bit dual-port SRAMs for CRC Tables and a 64 x 1 1 2 bit single port 

29 SRAM for Statistic information. 
30 
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1 Data Structure for Reassembly Table 

2 

3 See Figure 224. The RE AS SEMBLYT ABLE is a 64 x 1 8 bit internal SRAM 

4 addressed by the port ID (PID). The Reassembly Table has 64 entries, one for each port. 

5 o SOP: Start of Packet 

6 o EOP: End of Packet 

7 o Packet Length: A running count of the number of bytes in that packet. 
8 

9 Data Structure for OUTPUTCONTROLMEMORY 

1 0 See Figure 225 . The OUTPUT CONTROL MEMORY is a 1 28 x 1 8 bit internal 

1 1 SRAM that is used to store the control information for each cell that is ready to send to 

12 the SPI block along with the next read pointer and next number of Header words for the 
5 13 next cell linked to that port in the DATA SRAM and the number of valid bytes in that 
Nl 14 cell. 

J 15 o SOP: Start of Packet 

y 16 o EOP: End of Packet 

jH 17 o CELL LENGTH: Number of Data bytes valid for that cell. Used 

* n 1 8 internally to create the Byte valid signals to the SPI interface. 

2 19 (Combine with number of Header words for 

P 20 RAS SPI CELL LENGTH sent to SPI OUT.) 

|jj 21 o NEXT RDPTR: Pointer to the Head of the next cell of data queued 

22 to that port in the Data SRAM. 

23 o NEXTHDR WD: Number of header bytes for the next cell of data 

24 queued to that port in the Data SRAM. 
25 

26 Data Structure for OUTPUT_PORT_QUEUE 

27 See Figure 226. The OUTPUT_PORT_QUEUE is a 64 x 1 8 bit internal SRAM 

28 that is used to store the output port Queue Head (RD PTR) and Queue Tail (WR PTR) 

29 pointers for each cell that is ready to send to the SPI block along with the port Empty 

30 status and the number of Header words in the Queue Head cell. 
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1 o RDJPTR: Pointer to the Head of the next cell for that port that should 

2 be sent to the SPI Out. 

3 o WR_PTR: Pointer to the Head of the last cell for that port that should 

4 be sent to the SPI Out. 

5 o E: Empty - Status of cells queued and linked for that port. (Default = 

6 1, Empty) 

7 o PEND: Cell Pending - Cell is waiting to be queued to output port. 

8 (Default = 0, Not Pending) 

9 o HDR_WDS: Number of header words for the next cell for that port to 

10 be sent to the SPI Out. 

11 o 

1 2 Data Structure for FREE_BUFFER 

Q 1 3 See Figure 227. The FREE_BUFFER is a 1 28 x 7 bit internal SRAM that is used 

14 to store the pointer to the free data block in the DATA SRAM. A pointer is taken from 

^ 1 5 the top of this table when a data block is needed to receive the incoming data. When the 

S 1 6 SPI interface gets a complete cell off the queue the pointer to that cell is returned to the 

!f 17 end of this table. 

: . lg 0 FREEPTR: Pointer to the head of a 64-byte buffer in the Data 

% 19 SRAM. 

u 20 

□ 21 Data Structure for DAT A_SRAM 

*"* 22 See Figure 228. The DATA SRAM is used to store the data while reassembling 

23 it into the proper cells for transmission. There are 128 64-byte buffers. This can support 

24 128 64-byte cells. 
25 

26 Data Structure for HEADERJSRAM 

27 See Figure 229. The DATA_SRAM is used to store the data while reassembling 

28 it into the proper cells for transmission. There are 128 16-byte buffers. This can support 

29 128 16-byte Headers. 
30 
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1 Data Structure for CPU_DATA_SRAM 

2 See Figure 230. The CPU_DATA_SRAM is used to store cell data destined for 

3 the CPU. There are 4 72-byte buffers. This can support 4 64-byte Cells of Data plus 8 

4 bytes of control information. 
5 

6 Data Structure for CRC TABLE 

7 See Figure 23 1 . The CRC_TABLE is a 64 x 32 bit internal dual port SRAM that 

8 stores the partial CRC for each port reassembly. 

9 o PART CRC: Partial CRC stored after each cell for that port. Used to 
10 calculate the CRC for an entire packet. 

11 

12 Data Structure for STATISTICS 

13 See Figure 232. The Statistic SRAM is a 64 x 1 12 bit internal SRAM addressed 

14 by the port ID (PID). Each port has an entry to this SRAM to store the number of packets 

1 5 and data received for that port along with the number of CRC error packets. The content 

16 of this SRAM is accessed through the PCI interface block. 

17 o PKT_RCVD: Counter for the number of Packets received. 

18 o DATA RCVD: Counter for the number of Data bytes received. Only 

1 9 includes valid data bytes. 

20 o CRC_ERRORS: Counter for the number of CRC Errors received. 

21 o TTL ERRORS: Counter for the number ofTTL Timeout Errors. 
22 

23 Data Structure for EXTERNAL LOOK-UP MEMORY 

24 See Figure 233. The External Look-Up SRAM is 4M x 36 bits made up of eight 

25 5 1 2K x 36 bit external SRAMs addressed by the FLOW ID (FID). Each FLOW has two 

26 entries to this SRAM to store the Header data for that Flow. The Flow Header Data 

27 (FHD) is two or four 36-bit words that contain up to 2 MPLS tags, 1 VPI/VCI label, or a 

28 Layer 2 Header. On the Ingress chip a total of 12 bytes are available for new user 

29 defined headers with the remaining 4 bytes of Header data used for a Maximus internally 

30 defined Switch Header. On the Egress chip all 1 6 bytes are available for new user 

3 1 defined headers as our Maximus 4 byte header will not be append in this case. The two 
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1 control bytes contain a number of fields that indicate how to process the packet or cell. 

2 Each FHD is linked to a user defined Header, an MPLS tag or a VPI/VCI label through 

3 the FLOWED. 
4 

5 FLOW HEADER FORMAT: 

6 See Figure 234. The following fields are in the Flow Header Data Table: 

7 1 . 1 Byte Control word in 1 st FLO WID word to determines the type of FHD 

8 (MPLS, ATM or L2) and the MPLS operation to perform as well as the 

9 number of Header words . 

10 2. 32 ATM/32 MPLS bit New Label 1 

11 3. 32-bit MPLS New Label 2 

12 4. 1 Byte Control word in 2 nd FLOWED word to determine the number of L2 
O 13 Header bytes to be stripped. 

^ 14 5. 32 bits for possible Longer Header Formats. 

"2 15 6.32 bits for our internal Switch Header information. 

y i6 

U 17 CONTROL BYTE FORMAT: 

L 1 8 See Figure 235 . The first control word is bits 7 1 to 64 of the first 72-bit word of the 

m 19 FLOW ID with the second control word being bits 71 to 64 of the second 72-bit word for 

jest 

20 that FLOW ED. The following fields are in the Flow Header Control Words: 

Q 21 1. HDR TYPE = 2 bits that determines the type of Header (MPLS, ATM or 

22 Ethernet) 

23 a. 00 = MPLS 

24 b. 10 = ATM 

25 c. 01 = L2 

26 d. 1 1 = Reserved 

27 2. MPLS OPERATION = 3 bits to indicate the type of MPLS processing to 

28 perform. 

29 e. 000 = Replace MPLS Tag; Decrement MPLS TTL 

30 f. 001 = Push 1 MPLS Tag; Decrement original MPLS TTL 

31 g. 0 1 0 = Push 2 MPLS Tags; Decrement original MPLS TTL 
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1 h. 01 1 = Pop 1 MPLS Tag; Decrement original MPLS TTL 

2 i. 100 = Pop 2 MPLS Tags; Decrement original MPLS TTL 

3 j. The last 3 cases are reserved for future implementations. 

4 3. HDR = 2 bits to indicate the number of valid Header words. (00 = 0, 01=1, 

5 1X=2) 

6 4. L2EN = L2 CRC checking Enabled 

7 5. # BYTES L2 HDR STRIP = 4 bits for Number of L2 Header Bytes to strip if 

8 type is ETHERNET. 

9 6. RSVD = 4 bits Reserved for future use. 
10 

1 1 MAXIMUS 4 BYTE HEADER FORMAT: 

12 See Figure 236. The Ingress Maximus chip will append a proprietary 4-byte header 
Q 1 3 onto the end of any 8 or 1 6 byte switch or packet header before sending the data out. 

M 14 This allows the user to append a programmable 4 to 12 byte header along with the 

^ 1 5 Maximus 4-byte header onto every packet or switch cell that is transmitted from the 

y 16 Ingress Maximus chip. The Maximus header contains information of Flow ID, 

2 1 7 application types and Class, along with the OAM, CLP and EFCI bits for ATM cells. 

^18 o FID: FLOW ID 

m 19 o TYPE: Application Type 

j=f 20 o EFCI: Explicit Forward Congestion Indication 

Q 21 o CLP: Cell Loss Priority 

^22 o OAM: Operation, Administration and Maintenance 

23 o CLASS: Class of service 

24 o EOP: End Of Packet 

25 o SOP: Start Of Packet 

26 The FID and CLASS will be obtained from the Header Lookup memory while the 

27 remaining signals will be obtained from the Memory Manager interface. 



28 

29 ATM HEADER FORMAT: 

30 See Figure 237. 

31 o VPI: Virtual Path Identifier 
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1 o VCI: Virtual Channel Identifier 

2 o PT: PayloadType 

3 - EOP: End of Packet 

4 - EFCI: Explicit Forward Congestion Indication 

5 - MI: Management Information 

6 o CLP: Cell Loss Priority 

7 o HEC: Header Error Control 
8 

9 MPLS HEADER FORMAT: 

10 See Figure 238. 
11 

1 2 AAL5 TRAILER FORMAT: 

13 See Figure 239. 

14 o CPCS-UU: Common Part Convergence Sublayer 

15 o CPI: Common Part Indicator 

16 o L: Length of payload 

17 o CRC: Error detection in SSCS PDU (user data) 
18 

19 Interface 

20 System 

21 See Figure 240. 

22 Reassembly O Memory Manager 

23 See Figure 241. 

24 Reassembly O SPI 

25 See Figure 242. 

26 Reassembly OCPU 

27 See Figure 243. 
28 

29 TIMING 

30 See Figures 244-6. 
31 
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1 Memories 

2 Input phase to RAS from Internal Memory Manager. See Figure 247. Note: R2 

3 for the Partial CRC Table is dependent on the PID. If the next cell has the same PID as 

4 the previous cell then this read information is not used. 
5 

6 ERROR CONDITIONS 

7 Several errors can happen in the Reassembly engine with the error packets being 

8 marked during the EOP transfer to the SPI as EOP Abort. The different errors are listed 

9 in Figure 248. 
10 

11 Re gister Description 

n 12 

fees? 

^ 13 Registers 

H 14 See Figure 249. 

j 5 i 15 Commands: 

CS 16 See Figure 250. 

\ 17 

ssSe. 
: : 

'% 18 Commands 

M= 19 

20 Read external RAS memory 

21 See Figure 251. 
22 

23 Write external RAS memory 

24 See Figure 252. 
25 

26 FLOW HEADER FORMAT: 

27 See Figure 253. The following fields are in the Flow Header Data Table: 
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1 7. 1 Byte Control word in 1 st FLOWED word to determines the type of FHD 

2 (MPLS, ATM or Ethernet) and the MPLS operation to perform as well as the 

3 number of Header words. 

4 8. 32 ATM/32 MPLS bit New Label 1 

5 9. 32-bit MPLS New Label 2 

6 10. 1 Byte Control word in 2 nd FLOWID word to determine the number of 

7 Ethernet L2 Header bytes to be stripped. 

8 11.32 bits for possible Longer Header Formats. 

9 12. 32 bits for our internal Switch Header information. 
10 

1 1 Read Internal Data Memory 

12 See Figure 254. 
13 

H= 14 Write Internal Data Memory 

m 15 See Figure 255. 
U 16 

: 'i 
=== 

17 Read Four Internal Memories (Partial CRC1, Partial CRC2, Reassembly Table, 

18 Output Queue) 

19 See Figure 256. 
20 

2 1 Write Four Internal Memories (Partial CRC1, Partial CRC2, Reassembly Table, 

22 Output Queue) 

23 See Figure 257. 
24 
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1 Read PORT Statistics 

2 See Figure 258. 
3 

4 Write PORT Statistics 

5 See Figure 259. 
6 

7 Read CPU Data Memory 

q 8 See Figure 260. 

W 9 

W io Write CPU Data Memory 

==£ 

1 11 See Figure 261. 

Q 

S 12 

s 

R 1 3 Read 2 Internal Memories (Free Buffer List (FBL), Output Control) 

14 See Figure 262. 

15 

1 6 Write 2 Internal Memories (Free Buffer List (FBL), Output Control) 

17 See Figure 263. 
18 
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1 Read Internal Header Data Memory 

2 See Figure 264. 
3 

4 Write Internal Header Data Memory 

5 See Figure 265. 
6 

7 Get CPU Cell command 

== 8 See Figure 266. This command reads 64 bytes from the CPU Data FIFO 8 bytes 

9 at a time and stores them in the general purpose registers R0- R15. There is no address 
H 1 0 associated with this command as it just reads the CPU Data FIFO 8 times and then loads 

j~j 11 the Registers with the data. 

m 

3 i 

p= ^ 

3 

CP 13 Initialize Free Buffer List and Output Port Queue Memories command 

7; 

?=* 14 See Figure 267. This command initializes the Free Buffer List in the FBL 

1 5 Memory and sets the Empty and Pending bits to inactive in the Output Port Queue 

16 Memory. 

17 

1 8 Reserved Op-codes 

19 Op-codes 17 to 31 are reserved for future use. 
20 
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1 Principle of Operation 

2 The re-assembly block keeps track of a FLOW ID per port. The scheduler has a 

3 weighted round robin scheme that it uses to schedule a CELL for each port. If a port has 

4 more weight (or bandwidth) then it schedules more cells for that port. These scheduled 

5 cells are provided to the de-queue engine which sends the requests on to the Internal 

6 Memory Manager for pulling them from the SRAM. 

7 The Internal Memory Manager reads the scheduled cell from the external SRAM 

8 and stores it in an Output Data FIFO. Once the cell is stored in the Output Data FIFO the 

9 Internal Memory Manager queues the control information for that cell into the Output 

1 0 Control FIFO and asserts an available signal to the Reassembly Engine. 

1 1 The Reassembly Engine pops the Output Control FIFO to get the control 



12 information for the available cell. It uses the FLOW ID to look up the Header processing 

G 13 necessary for that cell as well as the Type field to determine the type of Reassembly to 

Cj 14 perform. 

^15 It also obtains a pointer to the Internal Dual Port Data SRAM from the FBL FIFO, 

y 1 6 Once it has started the Header look up and obtained a Free Buffer Pointer the Reassembly 

U 1 7 Engine starts reading the data from the Internal Memory Manager Output Data FIFO. 

^ 1 8 The data is read 128 bits at a time at 100 MHz. Each Data cell is 64 bytes long, with 

S 19 each data buffer being 64 bytes deep. There are 128 64-byte buffers in the Internal Data 

^ 20 SRAM. The Header information will be stored in a separate Internal Dual Port SRAM 

3™** 

0 21 that is also addressed from the FBL Pointer. The Header buffer has 128 16-byte buffers 

22 so it can handle up to 16 bytes per Header. The Header will be stored concurrently with 

23 the Cell Data being stored in the data buffer. 

24 Once the Header and Data are stored locally in memory the cell control 

25 information will be queued in the Output Control Memory and the Buffer pointer will be 

26 queued in the Output Port Queue Memory. If more than one cell is queued on a port then 

27 the Next Read Pointer will be stored in the Output Control Memory of the previous cell 

28 to link the current cell to the previous cell. 

29 The SPI has a weighted round robin scheme that it uses to Request a CELL for 

30 each port. If a port has more weight (or bandwidth) then it Requests more cells for that 

3 1 port. When the SPI requests a cell for a particular port the Reassembly checks the ports 
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1 Output Port Queue to see if there are any cells available for that port. If there are no cells 

2 available the Reassembly informs the SPI that the port has no data by asserting an Empty 

3 signal and the SPI continues on to the next port. If the port has cells available the 

4 Reassembly uses the Read pointer stored in the Output Port Queue to read the control 

5 information for that cell along with the associated Header and Cell Data. The 

6 Reassembly sends the Header and Cell Data to the SPI interface along with a valid signal 

7 and byte enables to specify the valid data bytes. The SPI will use the byte enables to strip 

8 any unwanted padding. After the Reassembly Engine sends the complete cell to the SPI 

9 interface it returns the Read Pointer to the FBL FIFO. 

10 The Reassembly engine has a Reassembly status table for each of the ports. It can 

1 1 simultaneously reassemble one packet at a time for each port. The local port Reassembly 

12 status table stores the control information for each port along with state information and 
□ 13 or any partial cell information. The control information tells the re-assembly block how 

5 14 to re-assemble the FLOW for that port, whether it is ATM to ATM, ATM to packet, 
^ 15 packet to ATM or packet to packet. 

yi 16 For AAL5 to Packet reassembly a CRC- 32 is calculated on the packet as each 

U 17 ce U is received. The partial CRC is stored on a per port basis and loaded back into the 

^ 18 CRC engine for each subsequent cell in that packet. Once it has received the end of the 

6 19 packet the calculated CRC is check against the received CRC to ensure packet integrity. 
H 20 If the CRC is in error the packet will be marked as bad by setting the Bad control bit in 
p 2 1 the Output Queue FIFO for the SPI interface. 

^ 22 For Layer 2 Packet reassembly the CRC-32 is checked on the packet as it is being 

23 sent to the SPI out interface. The CRC is calculated the same as the AAL5 CRC-32, but 

24 if there is an error detected it must be sent immediately to the SPI out as the complete 

25 packet has been sent in this case. The CRC checking is user selectable by programming 

26 the CRC enable bit in the Flow ID Lookup table. 

27 Statistics are kept on a per port basis for the number of Packets sent, the number 

28 of Bytes sent and the number of errors received. The CPU can read and write internal 

29 registers or statistics on a per port basis as well as access all internal memory locations 

30 and the external Header Lookup memory through the PCI interface. Data packets can 

3 1 also be sent to the CPU through the PCI interface. 
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1 

2 Initialization Setup 

3 The initialization process consists of writing the Initialize Memories command 

4 op-code to the Command register. The Reassembly block will then initialize the Free 

5 Buffer List and clear all the Empty and Pending bits in the Output Port Queue Memory. 

6 When the initialization process is done, the command op-code will be cleared to all 

7 zeroes to inform the system CPU that the initialization process is complete. 
8 

9 Re-assembly flow 

10 1 . Output scheduler selects the port. 

11 2. Priority scheduler for port decides on the FLOW ID. 

12 3. FLOW ID is sent to the De-queue engine. 

B 13 4. Output scheduler keeps the current head and other information per port. 

Si 14 5. De-queue engine queues the buffer pointers along with corresponding port # 

: 2 15 for Internal Memory Manager. 

W 1 6 6. Internal Memory Manager gives a data available signal to the reassembly 

U 1 7 engine when a complete cell is in the output data FIFO in the Internal Memory 

L 18 Manager. 

i 19 7. Internal Memory Manager sends ACK back to the scheduler 

J 20 8 . Scheduler releases the buffer from the INTERNAL MEMORY MANAGER 

P 2 1 and passes next port and buffer head to the de-queue engine. 

22 9. Internal Memory Manager tells Reassembly that a cell is available. 

23 10. Reassembly engine pulls the control information for that cell from the Internal 

24 Memory Manager control FIFO and then extracts the data from the Internal 

25 Memory Manager output data FIFO. 

26 11. Reassembly uses the control information to look up the Header processing 

27 necessary for that FLOW. It also obtains a free buffer pointer to the Data 

28 SRAM and uses the control information to determine the type of processing to 

29 perform. 

30 12. Reassembly engine stores the new Header information along with the data cell 

3 1 and Queues the cell onto the appropriate output port. 
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1 13. The SPIO requests data on a per port basis based on its own internal port 

2 calendar. 

3 14. Once the SPIO requests data for a port that has a cell queued on that port the 

4 Reassembly sends the control information for that cell along with the 

5 associated Header and Cell Data for transmission. 

6 15. The SPIO will strip extraneous pad data based on the byte enables that go with 

7 the data. 

8 1 6. The Reassembly engine sends CPU Buffer Full status information to the SPIO 

9 for combining with the port FIFO status information that is sent to the 
1 0 Scheduler from the SPIO. 

11 



M 14 

2 

m 
□ 

Lj. 
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1 

2 Per Flow Queue Engine 

3 The Per flow queue stores flows parameters and statistics in dedicated FID memory. 

4 In addition, it stores memory buffer pointers or IDs that hold data traffic of the flow IDs 

5 in a dedicated BID memory. Furthermore, it will retrieve the above FID parameters, 

6 statistics and memory buffer IDs when they are needed. It manages the free buffers list. It 

7 performs Random early discards on FID data cells. 
8 

9 Features 



10 • Supports 1M possible flows 

1 1 • Supports IK Tunnel Flows, 
y 12 • Supports IK Multicast Flows. 

Si 13 • Multicast one flow to 1 K flows maximum. 

2 14 • Collects statistics on a per flow basis. 
W 15 o Packet Counters: 32 Bits. 

2 16 o Byte Counters: 48 Bits. 

L 17 o Packet Discards per external Requests such as SPI: 16 Bits 

Pig o Packet Discards per Reassembly Timeout request: 1 6 bits. 

ssst. 

S 1 9 o Packet Discards per internal buffer management request: 1 6 bits. 

P 20 • Discards by returning buffer Ids to free buffer list. 

21 • Manages the Free Buffer List. 

22 • Manages RED by FID association. 

23 • RED Memory: IK x 48 bits. 

24 • FID Enqueue Memory: 2M x 72 bits. 

25 • FID Dequeue Memory: 1M x 36 bits. 

26 • FID Statistics Memory: 2M x 72 bits. 

27 • BIDLL Memory: 8M x 36 bits, including free buffer list. 

28 • Tunneling Dequeue Leaf Memory: IK x 56 bits, internal. 

29 • Tunneling Enqueue Leaf Memory: 2K x 82 bits, internal. 

30 • Tunneling Root Memory: 512x21 bits, internal. 
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1 • Multicast Dequeue Leaf Memory: IK x 36 bits, internal. 

2 • Multicast Enqueue Leaf Memory: 2K x 82 bits, internal. 

3 • Multicast Root Memory: 512x30 bits, internal. 

4 • Clock Frequency: 200 MHz. 
5 

6 Functional Description 

7 See Figure 278 . The Per Flow Queue Engine consists of the following blocks: 

8 Enqueue engine, Dequeue engine, Free List buffers manager, RED manager, Statistics 

9 engine, Timeout manager, and CIH Director. Enqueue engine places received data cell 

10 pointers on the queues. Dequeue engine retrieves and removes data cell pointers from the 

1 1 queues. The RED manager will track the total number of buffers on eight CLASS 

« 12 counters using each of the associated RED parameters. The cells will be dropped when 

1 3 the RED algorithms flags the cells as such. The statistics manager counts per flow 

Si 14 various parameters. The timeout manager discards packets or cells when they exceed a 

fl 15 certain timeout threshold on the enqueue side for the reassembly timeout. The timeout 

§ 16 manager discards packets or cells when they exceed a certain timeout threshold on the 

^ 17 dequeue side for the reassembly timeout. The PFQ interfaces to the memory manager, to 

0 1 8 the shaper and to the scheduler. The Per Flow Queue Engine handles the flow tunneling, 

5 19 discarding and multicasting of FID in addition to enqueueing and dequeueing the FIDS. 

£ 20 The statistics block will collect on a per flow basis the number of discarded flows, the 

21 number of bad flows, and the number of dropped flows because of RED. It will provide 

22 free buffers BIDS to the memory manager and it returns the buffer BIDS to the free 

23 buffer list. 
24 
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1 Block Description 

2 Algorithms 

3 

4 Enqueue Normal FID Algorithm 

5 The enqueue request will place a BID of an FID on the queue. It will modify the 

6 BID as well as the FID Enqueue memory. 

7 When the queue is empty, the first enqueue is for an SOP cell. If the cell is not SOP, then 

8 the cells are dropped until the next SOP. See Figure 279. 



9 1 . The FED Enqueue memory is read, the tail pointer will point to the last BID in the 

10 link. 

11 2. The BID tail is compared to "Null"; if it is a null then the queue is empty. 

12 3 . The current BID tail that was stored in the FID Enqueue memory BIDC will be 

13 used to access the BID memory. The new BID Tail is written as the content of 

14 that BIDC Tail location. If the cell is SOP or EOP then the flags are written in the 

1 5 BID memory with the BID link. 

1 6 4. The new BID replaces the tail pointer in FID Enqueue memory. 

17 5. The new tail is linked to "NULL" in the BID memory. 

1 8 • When the tail pointer is read, it will be checked to see if it is null. When it is null 

19 then the Dequeue memory has to be written with the new head and the associated 

20 flags such as the SOP, CLP, and EOP etc. 

21 • If an EOP is received, then a shape command is issued to the shaper. 

22 • The count is sent to the statistics block with the FID when the EOP flag is 

23 received. 

24 • The count is sent to the shaper and then it is not saved in the FID or BID 

25 memories. 
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2 Dequeue Normal FID Algorithm 

3 The de-queue request will remove a BID from an FID. 

4 1 . The FID Dequeue memory is read; the head BID pointer will point to the first 

5 BID in the link. 

6 2. The current BID head that was stored in the FID Dequeue memory will be used to 

7 access the BID memory. 

8 3. The new BID will replace the head pointer and the value is written in the FID 

9 memory. If the BID has other flags like SOP, EOP they are written at the same 

10 time as the BID head. 

1 1 • The new BID is compared to Null. If it is equal to null, then the Queue is empty 

12 and the Enqueue memory for that FID has to have the tail as Null. 

13 • A dequeue command is sent to the memory manager. 

14 See Figure 280. 
15 

1 6 Enqueue Multicast FID Algorithm 

17 When the queue is empty, the first enqueue is for an SOP cell. If the cell is not 

18 SOP, then the cells are dropped until the next SOP. The bit in the control register will 

19 determine the treATMent of multicast traffic. If the bit is set to treat the multicast traffic 

20 as Unicast, then the traffic is enqueued, as Unicast one. Else, the traffic is treated as 

21 described in the following section. The multicast control bit in the control register is 

22 reset. The PFQ will discard all the enqueue requests from the memory manager when an 

23 EOP cell is received for a multicast traffic. When the SOP cell is received, it is enqueued 

24 as a Unicast cell. The buffer ID pointers are linked in the memory. The pointers in 

25 memory are enqueued only once. The count is read from the Multicast Root memory. 

26 Every BID will have the count of the previous BID. When an SOP cell is enqueued, the 

27 tail pointer is in the enqueue memory. There is no link in the BID memory to the tail 

28 pointer in the enqueue FID yet. When a NULL is written to next location, the count is 

29 properly written in the correct field. When the EOP cell is received, then the BID link is 
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performed as it was done for the previous cells. When the BID of the EOP cell is linked, 
the multicast enqueue operations has to link the head and tail BIDS to the various FIDS 
that are members of the multicast group. At this time, the enqueue engine will discard all 
the cells with the enqueue request from the memory manager until the multicast packet 
has been linked to all the FIDS in the multicast group. The multicast engine will perform 
the rest of the operations to enqueue the packet on the rest of the FIDS. The multicast 
engine will read the next FID that in the multicast group. The first FED member has 
already the pointers linked in its location. The multicast engine will update the head and 
the tail pointers in the enqueue and dequeue memories locations that correspond to the 
FID members if the FID queue is empty. If the FDD is not empty, then the tail pointer is 
updated only. When the last FID member is update it, then the enqueue engine will 
continue to enqueue rather than discard the cells. The process is not efficient and the 
performance suffers considerably. 

The buffer management whether it is RED or class will be performed on the first SOP 
when the cell is enqueued for the HEAD member of the multicast group. Thereafter there 
are no checks done. After receiving the EOP cell, the DBS will get the FIDS that are 
members of the multicast group only. 

1 . The packet is enqueued as Unicast one until the EOP cell. 

2. When EOP cell is received, then the packet has to be linked for the rest of the FID 
members of the multicast group. 

3. The FID root memory is read to retrieve the head FID, tail FID pointers. 

4. The head FID has the packet already linked on its queue. The next FID field in the 
multicast group is read. 

5. The next FID is compared to the tail FID to determine the last FID in the 
multicast group. 

6. The LEAF memory of the FDD in the next FID field is read to determine if the 
queue is empty. If the queue of the FID is not empty, then the tail pointer is 
updated it with the last pointer of the EOP cell. The new tail pointer is linked to 
the old tail pointer in the BED memory and the COUNT is written there, too. 
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1 7. If it is empty, then the tail pointer in the enqueue memory is updated it from the 

2 enqueue memory. The head pointer in the dequeue memory has to be updated it 

3 from the dequeue memory. The leaf memory is updated it with BID pointers. 

4 8. The next FID in the FID leaf memory is compared to the tail FID in the root 

5 memory and the above operation is repeated until the next FID and the tail FID 

6 are equal. Then it is the end of the multicast group. 

7 9. The PFQ_MEM_FULL signal is asserted to the memory manager. 
8 

9 Dequeue Multicast FID Algorithm 

1 0 The dequeue command for a multicast FID is similar to the Unicast FID. They BID 

1 1 link is read from the BID memory and it will be the new head pointer. In addition the 

12 counter is read from the BID location that is the new head pointer. The counter is 

O 1 3 decremented and then written back in the same location with that BID. If the counter 

3 14 value is zero, then the "REL" bit in the command to the memory manager is set. Else the 

15 "REL" bit is reset and there will be no release command for that BID from the memory 

16 manager. If the BID link in the BID memory is "NULL", the queue for that FID is 

17 empty. The "NULL" has to be written back in that locations with the counter value equal 

1 8 to the previous count but decremented by 1 . 

19 1 . The FID leaf memory is read. 

20 2. The location read is the FED that is part of the dequeue command. 

21 3. The head BID pointer in the FID memory will point to a location in the BID 

22 memory. 

23 4. The location contains the next BID head pointer and the count. 

24 5. The count is decremented by one and written into memory. 

25 6. The new BID will replace the head pointer and the value is written in the FID leaf 

26 memory. 

27 7. The "REL" bit in the command is set when the count is zero. 

28 8. When the pointer is "NULL", it is handled the same way as in the Unicast FID. 
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1 

2 Enqueue Tunneled FID Algorithm 

3 When the queue is empty, the first enqueue is for an SOP cell. If the cell is not SOP, 

4 then the cells are dropped until the next SOP. The enqueue of the tunneled FID is exactly 

5 as the normal FID. When an SOP cell is received, the tail and head pointers are updated it 

6 as usual. When the EOP cell is received, then the FID that the PFQ sends to the DBS is 

7 not the same FID that was used to enqueue the packet. Instead, the FID will be read from 

8 the Leaf memory from a memory location pointed to by the FID of the enqueue 

9 command and it is in the ROOT Tunneling field. The FID that was read as the Tunnel 

10 ROOT memory represents one or more FIDS. The DBS will not know about the 

1 1 members of the ROOT FED. The FID in order to be tunneled has to be a member of that 
q 12 specific tunnel. 

1 3 1 . The leaf FID memory is read, the tail pointer will point to the last BID in the link. 
SJ 14 The root memory association is read. 

^15 2. The current BID tail that was stored in the FID memory will be used to access the 

W 1 6 BID memory. In that location the new BID is written. 

s 17 3 . The new BID replaces the tail pointer and the FID location is written to the leaf 
1 18 FID. 

y 19 4. The ROOT memory corresponding location is read. 

E 20 5. If the not empty flag is reset, then the LEAF FID is written as the "CURRENT 
^21 FID HEAD" and as the Tail FID 

22 6. In EOP cell cases, the ROOT FID is sent to the DBS with the cell count. 

23 

24 Dequeue Tunneled FID Algorithm 

25 The dequeue command that the DBS send to the PFQ contains an FID. The FID is the 

26 ROOT FID. The root memory has to be read and the member of the tunnel group has to 

27 be found. The Root memory has the head of the FID. The head is read and the FID leaf 

28 memory is read. After reading the Leaf memory, the dequeue operation proceeds the 

29 normal way. When the cell is EOP, then the "NEXT FID" field is used to update the new 

30 FID head pointer. If the next FID queue is empty, then the next FID field in that FID 

3 1 entry will be used and so on. 
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1 . The FID Root memory is read; the current FID pointer of the tunneled FID is 
retrieved. 

2. The FID leaf memory is read and the current BID head that was stored in the FID 
memory location will be used to access the BID memory. 

3. The new BID will replace the head pointer and the value is written in the FID leaf 
memory. 

4. The LEAF FID and the BID are sent to the memory manager. 

5. If the BID belongs to an EOP cell, then the "NEXT FID" field is read from the 
leaf memory. 

6. The previously read "NEXT FID" field will be used to update the "CURRENT 
FID TUNNEL HEAD" in the root memory. 

7. The new FID head leaf memory entry is read to determine if the queue is not 
empty. 

8. If the queue of the head FID is empty, then the "NEXT FID" field of that FID is 
read and the process is repeated until an FID without an empty queue is reached. 

9. During 

10. During the process, if the tail FID is reached, then there are no more BIDS on any 
queue. 

1 1. At that point the process terminates until the next dequeue command. The "NOT 
EMPTY" flag is reset. 

12. During the above process, assert "PFQ_ DBS_FULL" to the DBS. 

Enqueue Engine 

See Figure 181. The enqueue engine gets its commands from the memory. The 
enqueue engine issues commands to the shaper. If there are no free buffers or the request 
is to discard the FID, then the "NULL" FID is returned. If there is an enqueue request and 
there are no free buffers available, then the "NULL" BID is sent to the memory manager. 
If the RED determines that the packet has to be discarded, then the "NULL" BID is sent 
to the memory manager. 

The enqueue engine will enqueue the FID, discard the BID marked for discard, 
enqueue the multicast FID, and enqueue the Tunneling FID. The enqueue engine has to 
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1 issue the FID and the BED memory commands always within the eight cycles window. 

2 The FID of the enqueue and de-queue commands will be compared and checked if they 

3 are the same. If they are the same FID, then only the last FID memory write is valid and 

4 has the correct information. The Enqueue information will be merged with the dequeue 

5 information and written during the dequeue operation. The BID memory accesses depend 

6 on the FID results. The data coherency of the BID during multicast dequeue operations 

7 has to be maintained by the logic with respect to the BID content of that location and its 

8 count in case of multicast. 

9 The Tunneling of FIDS as well as the multicasting of FIDS is based on a circular 

1 0 buffer. Once the last FID is reached, then the enqueueing engine will point to the first 

11 FID in the link. Tunneling uses the root FID in the enqueue as well as in the dequeue 

12 process. The Shaper has no knowledge of the FIDS that are members of the Tunneled 

1 3 FID. The shaper has no knowledge of the multicast root FID. 

14 When the memory manager issues a command to the enqueue engine, the FID 

1 5 parameters are retrieved and the BID is enqueued. If the EOP is set, then a shape 

1 6 command is sent to the shaper. If the FID was empty, then the FID is activated. 

17 1. The enqueue engine will read the following: 

1 8 • FID parameters from FID enqueue memory. 

1 9 • Free buffer head from the free buffer register. 

20 • RED counter. 

21 2. The enqueue engine will read the following: 

22 • Read the BID memory location FBH to get the new FBH. 

23 3 . The corresponding statistics from statistics memory of the FID. 

24 4. The enqueue engine will perform the checking operations of the various 

25 conditions. The counters are incremented. 

26 5. The enqueue engine will write the following: 

27 • The BID memory location current BID Tail with BID as content. 

28 • The BID memory location new BID tail with the "NULL" as content. 

29 • The "FID" enqueue location with the new parameters. 

30 • The "FID" dequeue location with the new tail if the queue is empty. 

3 1 • The FID statistics location with the new statistics. 
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1 • The RED Counter with the new value. 

2 • The Free buffer register with the new head. 
3 

4 Dequeue Engine 

5 The Dequeue engine gets its commands from the scheduler. The Dequeue engine issues 

6 commands to the memory manager and to the enqueue engine. When there is a dequeue 



7 command, the PFQ will get all the parameters from the FID and sends a command to the 

8 memory manager for retrieval of data. Then the PFQ will update the buffers pointers 

9 See Figure 282. 

10 1 . The dequeue engine will read the following: 

1 1 • FID parameters from the Dequeue FID memory. 

12 • The BID memory location BID head. 

13 2. The de-queue engine Will perform the checking operations of the various 

14 conditions. The head pointers are updated. If the head is null then the queue is 

15 empty. 

16 3. The dequeue engine will write the following: 

1 7 • The "FID" dequeue location with the new parameters. 

1 g • The "FED" enqueue location with the new tail "NULL" if empty. 
19 

20 Release BID 

21 The free buffer engine will return the BID to the free buffer list. It can be 

22 simultaneously issued with the enqueue request. The bit "REL" will determine if the 

23 memory manager will release the BID or not. 

24 • The free buffer BID memory with the new link BID. 
25 

26 Free buffer manager 

27 There is a free buffers list. There is a corresponding head and tail registers on the 

28 device. There is a link list linked in the BID memory. The link list is initialized on init 

29 command. The enqueue will use the head of the free buffers. The tail of the free buffer 

30 list is updating it when the enqueue and dequeue engines are accessing it. The buffer 

3 1 number "7FFFFF" is reserved for not valid BID. The cell should be dropped when the 
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1 memory manager gets this value in case of a discard request from the memory manger or 

2 the RED algorithm. See Figure 283. 

3 There are three parameters that are stored in registers for the free buffer list. 

4 • The number of available buffers. It represents the raw total storage memory space. 

5 • The threshold of no buffers. 

6 • The number of used buffers. 

7 The following operations are done on the counters: 

8 • Whenever a buffer is enqueued, then the number of used buffers is incremented. 

9 • When a buffer is released, then the number of used buffers is decremented. 

1 0 • The number of used buffers can never be less than "0". 

1 1 • The number of used buffers can never be more than the available buffers. 

12 When the number of used buffers is equal or larger to the threshold of number of buffers, 
% 13 then: 

Mas 

^ 14 • All the queue of that cell has to be emptied and all the buffers of that queue are 
yg 1 5 returned to the free buffers list. The buffers in use counter will be decremented by 

16 . the number of cells. 

^17 • Update the free buffer register and memory link in the BID memory. 

0 18 

19 Timeout manager 

20 It is running with simultaneously with the enqueue operation. The timeout 

21 manager is used to remove orphan cells for reassembly timeout and free the storage for 

22 incoming traffic. When an FID corresponding memory location is read for an enqueue 

23 operation, there are two TTL bits that are reset to "00" when the FID memory location is 

24 written back. The memories that are accessed are the external FID memory and the 

25 internal leaf FID memories. The internal memory will be accessed once the external 

26 memory has been completely accessed once. The algorithm is not a fair one. When the 

27 timeout manager reads a location the TTL bits are changed from "00" to "01" or changed 

28 from "01" to "10". If the FID location is read and the TTL bits are "10", then there is a 

29 reassembly timeout and the packet remnants are discarded. The Qsize has to be 

30 decremented whenever there is a discard. It should have the value "0" in the statistics 

31 memory. 
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The statistics memory location is read. Every 8 cycles or every time slot, one 
location of the enqueue well as the statistics memories is read to perform the timeout 
operations. If there is a reassembly timeout, then the start of timeout operation is 
suspended until the data is updated in both the enqueue and the statistics memories. 
The timeout manager will do the following: 

• An active address that points to an FID memory location. The address is 
sequential and circular. It is updated when the access is executed. 

• Every sync cycle the FID memory is accessed, a location corresponding to that 
address is read and examined. 

• If the TTL bits value is "00", then the value will be changed to "01". 

• If the TTL bits value is "01", then the value will be changed to "10". 

• If the TTL bits value is "10", then it is reassembly timeout and the packet chunk 
is discarded on the enqueue side. 

• The TTL bits value then are reset to "00"after the discard. 

• The DROP timeout bit is set. All the fragments of a packet has to be dropped 
until the next SOP. At the next SOP enqueueing starts again. 

DISCARD FID 

The discard operation is running in place of the enqueue operation. Whenever 
there is a discard because of Reassembly timeout, the Qsize has to be decremented. It 
should have the value "0" in the statistics memory. There are several discard commands: 

• Memory manager generates a discard command of a packet that is being 
enqueued. 

• RED manager generates a discard command of a packet violating the algorithm. 

• Free Buffer manager generates a discard command of a packet being enqueued 
because of lack of free buffers. When the number of used buffers exceeds a 
programmed backpressure threshold register, the enqueue engine will signal the 
segmentation the above condition. If the number of used buffers drops 10 buffers 
below the programmed backpressure threshold, then the PFQ will signal the 
above message to the segmentation engine. If number of used buffers exceeds the 
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programmed threshold register then the enqueue engine will discard these buffers 
until the next EOP. The PFQ will discard the packet fragments from the queue in 
such a condition. 

• Timeout manager generates a discard command when the packet is not 
completely enqueued because the later chunks were not received or got dropped. 
The packet is a tail of the queue of that FID. 

• CLASS discard; the packet will be discarded when the queue counter is above the 
programmed threshold. On every SOP the CLASS counters are checked. 

• Port Blockage; the packet will be dropped from SOP to EOP until the bit is 
changed in memory. On every SOP the port register is checked. 

When the memory manager generates the discard command, then the following steps are 
taken: 

1 . The FID enqueue memory is read. The previous packet tail is parsed. 

2. The BID memory is read and the BID of that location is read. The content is the 
head of the new packet being enqueued. 

3. The Free Buffer tail is read. 

4. The BID memory location of address FBT will have the content the head of the 
packet that is being discarded. The buffers are on the free buffers list. 

5. All the buffers are returned to the free buffer list. 

6. The FID memory location is updated with the new tail pointer, which is the tail of 
the FID. 

When the free buffer manager generates the discard command, then the following steps 
are taken: 

• The cell is dropped and the BID is returned to the free buffer list. There is no 
other operation required. 
When the timeout manager generates the discard command because of reassembly 
. timeout, then the following steps are taken: 

1 . The FID memory is read. The previous packet tail is parsed. 

2. The BID memory is read and the BID of that location is read. The content is the 
head of the new packet being enqueued. 

3. The Free Buffer tail is read. 
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1 4. The BID memory location of address FBT will have the content the head of the 

2 packet that is being discarded. The buffers are on the free buffers list. 

3 5. All me buffers are returned to the free buffer list. 

4 6. The FID memory location is updated with the new tail pointer. 

5 When the RED manager generates the discard command, then the following steps are 

6 taken: 

7 1. The cell is dropped. 
8 

9 Port Management 

1 0 On each SOP enqueue, the enqueue engine will check the port register to 

1 1 determine if the packet has to be dropped or not. The port number is available from the 

12 statistics memory. If the port register indicates that the cell has to be dropped, then the 

1 3 SOP cell is dropped. If the cell is not SOP, then the drop packet flag in the FID memory 

14 is checked. If the flag is set then the cell is dropped, else the enqueue operation continues. 

1 5 The port packet discard flag for that FID is set. The flag will be reset when the port 

16 register indicates that the port is re enabled again. 
17 

1 8 CLASS buffer Management 

1 9 Enabling the buffer management bit in the control register will enable either the 

20 RED algorithm or the CLASS management. If the RED bit is reset, in addition to the 

21 previously enabled bit, then the CLASS management algorithm is enabled. 

22 The enqueue engine will increment the counter in the CLASS register corresponding to 

23 that FID on every cell. The Dequeue engine will decrement the counter on dequeue 

24 command. When enqueue an SOP cell, the counter of the corresponding class. 

25 On each SOP enqueue, the class counter will be compared to the threshold counter of that 

26 class. If the counter value exceeds the threshold, then the SOP cell is dropped. The cells 

27 of the rest of the packet are dropped until the next SOP cell. At that point the counter 

28 value is checked against the threshold and the process is repeated. 
29 
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1 Random Early Discard Management 

2 Enabling the buffer management bit in the control register will enable either the 

3 RED algorithm or the CLASS management. If the RED bit is set in addition to the 

4 previously enabled bit, then the RED algorithm is enabled. Another way to bypass the 

5 RED is by making both thresholds equal and very high. The average queue will not get as 

6 high as the threshold. The drawback is the calculations are still needed. The queue size 

7 and the averages are calculated according to cell size. The Queue size is updated every 

8 time there is an EOP. It is incremented by the number of cells. The queue size is 

9 decremented every time there is a dequeue operation. It is decremented by one. 

10 When an SOP of an FID is received, the RED algorithm decides if the packet gets 

1 1 discarded. If the packet is discarded, then the SOP cell is dropped as well as subsequent 

12 arriving cells of the packet are dropped until the EOP cell is received. The BID is 
O 13 returned to the Free Buffer List. 

5 14 With constant average queue size the goal is to discard packets at regular intervals 

^15 in order to avoid clustering of discards and global synchronization. It is not desirable to 

l5 16 have too many discarded packets close together and also it is undesirable to have a long 

y! 17 interval between discarded packets. 

^ 1 8 The RED algorithm calculates the queue average size when an SOP cell is 

51 1 9 received and compares it to two thresholds. If the size is less than the minimum 

hf 20 threshold, then the packet is not dropped. If the queue is larger than the maximum 

S 2 1 threshold then the packet is dropped. If the average size is between the minimum and the 

22 maximum thresholds, then a discard probability is calculated to determine the fate of the 

23 packet. There is a random number that will be used to determine if the packet has to be 

24 discarded in this case. 

25 The first step when a packet is received by an empty queue is to estimate the 

26 number of packets that would have been transmitted during idle time. Then the average 

27 queue size will be calculated using the original estimate as if the number of packets has 

28 been transmitted during that interval. The average queue size is calculated for every FID. 

29 There is 1 K parameter memory for the various attributes such as the minimum and the 

30 maximum thresholds and the weight of the queue. There is an association between the 

3 1 FIDS and the attributes in the parameter memory. 
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The MAXth is decreased when the probability P a of discard is increasing. When 

2 the probability changes slowly the queue average size changes slowly. This will 

3 discourage oscillation in the average queue size. The values of attributes MINth and 

4 MAXth determines the utilization of the bandwidth 

5 RED algorithm needs the following fixed and calculated parameters. 

6 The following variables are calculated and saved: 

7 • "AVG": The average queue size. 

8 • «q_ time "; The start of queue idle time. 

9 • "COUNT": The number of not discarded packets that have arrived since last 

10 discard. 

1 1 • "Q": It is the current queue size. It is incremented on enqueue with the number of 

12 cells and decremented with one on dequeue. 

1 3 The following parameters are fixed and programmable: 

14 • "W q ": It is the queue weight. An initial value of 0.002 is used. It should not be 

1 5 large in order to filter transient congestions. 

16 • "MIN th ": It is the minimum threshold for queue. If the traffic is bur sty, then the 

1 7 value has to be fairly large. 

18 • "MAXth": It is the maximum threshold for queue. The value determines the 

19 maximum average delay in the device. An initial value of 3 * MIN t h is used. 

20 • "MAXp": It is the maximum value for a probability "P b " of discard. An initial 

21 value of 1/50 is used. It should never be greater than 0. 1 . 

22 • "S": Typical transmission time for a small packet. 

23 • "RANDOM": Random number from generator. 

24 The following variables are calculated but not saved: 

25 • "P a ": It is the current packet discard probability. 

26 • "TIME": It is the current time. 

27 • "F (/)": It is a linear function of time t. 

28 • "M": The number of packets that would have been transmitted when queue is 

29 empty. 

30 The following calculations are needed: 
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1 • The number of packets "m" that might have been transmitted during idle time. 

2 • The average size of the queue. 

3 •The probability of discard when the queue size is between the minimum and the 

4 maximum thresholds. 
5 

6 Equations 

7 The queue empty size "M" equation is the following: 

8 M = (TME-Q- TIME )/S 

9 The packet discard probability P b equation is the following: 

10 P b = MAXp ( AVG - MIN, h )/(MAX, h - MiN, h ) 

11 It is desirable to have P b a negative power of 2. 

1 2 The final packet discard probability P a equation is the following: 
g 13 P a = P b /(1- COUNT *P b ) 

H 14 The average queue size is calculated according to the following: 

2 15 AVG = (l-W q )*AVG + W q *Q 
W 16 

m 

M= 17 Implementation 

^ 1 8 It is an easy determination when the average queue size is larger than MAX th and 

§19 is smaller than MIN, h . The most complex part is when the average queue size falls 

U 20 between the MTN th and the MAX th . When the queue average size is less than MIN th , then 

2 2 1 the packet is transmitted. On the other hand when the queue average size is more than 

22 MAX t h then the packet is discarded. If the queue average falls between the MIN th and 

23 MAXth then the probability determines the packet destiny. A pseudo random number 

24 "R" is generated by a uniform distribution random numbers generated. 

25 The packet is discarded if "R" satisfies the following equation is satisfied: 

26 R < P b /(l- COUNT *P b ) 

27 If the average queue remains constant then the random number "R" is selected 

28 from the uniform distribution. The N, h packet is discarded if the following equation is 

29 satisfied: 

30 R<P b N 
31 
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1 Random Selection 

2 The random number is generated using linear feedback shift registers. The 

3 Ethernet polynomial will be used to generate a random number. The number is generated 

4 every time there is an enqueue operation. The following polynomial is used to generate 

5 the random number. 

6 G(X) = X 32 + X 26 +X 23 +X 22 + X l6 + X ,2 + X n + X ,0 + X 8 +X 7 + X 5 +X 4 +X 2 + X l +l; 

7 The value of the shift register is used as the number generated. The number of 

8 nibbles that is stored in the attribute table is used to determine the random number is 

9 used. If the value stored is "FF", then all eight nibbles are used. If the value is "7F", then 
10 the first 7 nibbles are used. See Figure 284. 

11 

12 Memory Size & Total Number of BIDS 

13 See Figure 285. 



1 5 BID Memories Chip Select 

16 See Figure 286. 



L 18 EXTRA MEMORY CYCLES 

y 

P 1 9 The extra needed cycles of the FID memory are the following: 

o 

y. 20 • If the queue becomes empty after a dequeue operation the tail in the enqueue 

H 21 memory has to be written as "NULL". 

bssz 

22 • If the queue is empty before an enqueue operation the head in the DEQUEUE 

23 memory has to be written with the same value as the tail pointer. 

24 • The Dequeue operation has to decrement the Qsize in the statistics memory. 

25 The BID memory requires an extra cycle after a tail is linked in memory. The extra 

26 cycle will link the tail to a "NULL". When the head reaches a "NULL" then the queue is 

27 empty. 
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1 

2 Memories Byte Enables 

3 The state machines have to drive the byte enables to the various memories. There 

4 are some write cycles during which only selected byte enables have to be asserted. The 

5 rest of the byte enables should not be asserted. 

6 The assertion of selected byte enables during write operations eliminates the need for 

7 a read operation. 

8 • Read Operations for all memories: The state machines will assert all the byte 

9 enables during all read operations. 

10 • Write Operations: 

1 1 o Enqueue Memory: When the queue becomes empty after a dequeue 

q 1 2 operation, the tail pointer in the enqueue memory for that FID has to be 

;J1 1 3 written as NULL. When writing that location, the byte enable to bits 27 

si 14 downto 0 will only be asserted. 

15 o Dequeue Memory: When the queue is empty and the there is an enqueue 

09 16 operation, the head pointer in the de-queue memory for that FID has to be 

' a 1 7 updated it. When writing that location, the byte enable to bits 36 downto 0 

jj i g will only be asserted because the TYPE comes with the FID. 

S 19 o BID Memory: When the NULL pointer is written at the end of an enqueue 

« 20 operation, the BID memory location has no relevant data. All byte enables 

*■* 21 will be asserted. 

22 o Statistics Memory: The dequeue engine has to decrement the Qsize of that 

23 FID after a dequeue operation. The STAT memory is read to get the value. 

24 Therefore, the all byte enables are asserted. 
25 

26 Global Synch and internal Time slot generation 

27 The global synch is provided for all the blocks. The block has to generate locally 

28 eight time slots. The memories controllers access these memories only during certain 

29 time slots. It is important that the all tasks are completed in one cycle of eight time slots 

30 or clocks. The tasks can take longer than eight cycles to complete. But the caveat is that a 

3 1 task has to start every eight cycles or time slots. The latency is longer but the bandwidth 
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1 is the same. The scenario complicates the data dependency of all the concurrent tasks and 

2 sequential tasks. 
3 

4 External and Internal Memory Initialization 

5 On power up the memories are not initialized. There is an initialize signal that 

6 when is set by the PCI, it triggers state machine that will write a zero pattern in all 

7 memory locations of all memories except the BID memory portion of the free buffer lists. 

8 In the free buffer lists, the locations will be initialized to a value that reflects the previous 

9 BIDS . The first location of the free buffer list will have the value X"00000000". The 

10 second location will have the value X"00000001". The rest of the locations will have an 

1 1 incrementing values. These will be the free buffer values that are used for BIDS. The first 
— 1 2 BID "000000" is not used. It is the "NULL" pointer. 

i 13 

Cj 14 Data Dependencies and Coherency 

J3 15 The data and the FID of both the enqueue and dequeue previous operation that has 

U 

09 16 just finished have to be saved. The current FID has to be checked against the previous 

** 1 7 ones. If the current FID is the same as the previous one of either the enqueue and dequeue 

3 

O 1 8 operations, then the data that was saved will be used. The data in memory is not the 

jjg 19 correct one. 

20 There are several scenarios where the data coherency will get compromised. 

Q 

M= 2 1 The data coherency will get compromised with consecutive enqueue operations to same 

22 FID: 

23 • When the consecutive enqueue operations are for the same FID the statistics 

24 counters should be updated form internal registers. 

25 • When the consecutive enqueue operations are for the same FID the tail pointer 

26 should be updated from internal register. 

27 • When the consecutive enqueue operations are for the same FID the RED 

28 Calculations should be based on data in internal registers. 

29 • When the consecutive enqueue operations are for the same FED the Q Size should 

30 be incremented based on data in internal register, then written in the statistics 

31 memory. 
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1 The data coherency will get compromised with consecutive dequeue operations to same 

2 FID: 

3 • When the consecutive dequeue operations are for the same FID the head pointer 

4 should be updated from internal register. 

5 • When the consecutive dequeue operations are for the same FID the Q Size should 

6 be decremented based on data in internal register then written in the statistics 

7 memory. 

8 The data coherency will get compromised with enqueue and dequeue operations to the 

9 same FID: 

10 • The only time that is compromised when there is an enqueue and a dequeue 

1 1 operation to the same FID. If the dequeue operation renders the queue empty and 

1 2 a "NULL" has to be written as the tail in enqueue memory and the head in the 

1 3 dequeue memory. Instead with the enqueue operation, the queue will not be 

1 4 empty any longer. The enqueue operation will preempt the dequeue operation and 

1 5 the new BID will be the tail and the head pointer. 

16 -The Qsize in the statistics memory has to be update correctly. The number of cells 

17 is decremented by one before it is added to the Qsize. 

1 8 The data coherency will get compromised with enqueue and timeout operations to the 

19 same FID: 

20 • The data written to the FID memory has to be the enqueue one. The TTL 

21 increment and the discard operation have to be bypassed. There will be no 

22 discards of the queue. The TTL will be written as "00". 

23 See Figure 287. 

24 Statistics 

25 The statistics memory is accessed when there is an enqueue, dequeue and CPU 

26 operations. All the requests are issued simultaneously. 

27 The enqueue will update the various counters depending on the command issued by the 

28 memory manager. 

29 • Enqueue command will increment by the number of cells the Q Size counter on 

30 EOP. 

3 1 • Enqueue command will increment the packet counter on EOP. 
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1 • Enqueue command will increment the byte size counter on command. 

2 • Discard command will increment the others discard counter. 

3 The dequeue command will decrement by one the Q Size counter always. 

4 The RED will increment the RED discard counter if the cell is dropped on EOP. 
5 

6 NULL Pointer 

7 The value of "7FFFFF" is the "Null" pointer when the BID memories are all 

8 populated. 
9 

10 Shaper and Scheduler Commands flow 

1 1 When it receives an EOP of an FID, then the PFQ will send a shape command to 

12 the shaper with the cell count of that packet. The Cell or packet indication has to have the 
4 13 correct value. 

%| 14 in case of ATM cells, the PFQ will send to the shaper a command to shape the 

2 15 FID cell. 



W 16 

U 17 External Interface 



JU 1 8 The external interface will be a standardized interface complying with the global 

5 1 9 SYNC. There are four interfaces between the PFQ and the external world. External 

U 20 memories are part of the PFQ and the interface is considered internally for all practical 

!^ 21 purposes. 

22 • Enqueue a FID. The memory manger drives the request. The PFQ will respond 

23 with a BED and a valid signal. 

24 o The memory manager to PFQ command is issued in CLK 6. 

25 o The PFQ to memory manger response is issued in CLK 6, next synch. 

26 • Discard a FID. 

27 o The memory manager to PFQ command is issued in CLK 6. 

28 • Release BID. 

29 o The memory manager to PFQ command is issued in CLK 6. 

30 • The second interface between the PFQ and memory manager is to dequeue an 

3 1 FID. The PFQ drives the dequeue request to the memory manager. 
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1 o The PFQ to memory manager command is issued in CLK 6. 

2 • There is an interface between the PFQ and the shaper. The PFQ drives the request 

3 to the shaper. 

4 o The PFQ to shaper commands are issued in CLK 6. 

5 • There is an interface between the PFQ and the scheduler. The scheduler drives the 

6 requests to the PFQ. The request is to dequeue a BID of an FID. 

7 o The scheduler to PFQ command is issued in CLK 7. 
8 

9 Memory Structure 

10 

1 1 RED Attribute Memory 

12 This is internal memory. There are IK entries of this memory. The software writes 
5 1 3 it only. The address is the "RED_ASSOCIATION" from the FID memory. See Figures 
^ 14 288 and 289. 

J 15 

^ 16 Statistics Memory 

5 1 7 This is an external memory. There are 2 MEG locations. There are 2 locations per 

L 18 FID. The software writes only the port number. The rest is written by the logic. The CPU 

5 19 reads the counter values. The address is the "FID" for bits 20 downto lwith bit 0 as "0" 

S 20 or"l". 

S 21 See Figures 290, 291 and 292.. 

22 All discard counters are updated on EOP. If the packet is being dropped because 

23 of RED and the EOP cell has a discard command instead of enqueue, then the RED 

24 counter is not update it. 
25 

26 FID Dequeue Memory 

27 This is an external memory. There are 1 Meg locations. There is one location per 

28 FED. The software does not write any location. The software under normal operating 

29 traffic conditions does not need any values from this memory. The address is the "FID". 

30 See Figures 293 and 294. 
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1 

2 FID Enqueue Memory 

3 This is an external memory. There are 2 MEG locations. There are 2 locations per 

4 FID. The software writes only the RED association. The software under normal operating 

5 traffic conditions does not need any values from this memory. The address is the "FID" 

6 for bits 20 downto lwith bit 0 as "0" or "1". See Figures 295, 296 and 297. 
7 

8 Buffer ID Data Structure 



9 This is an external memory. There are 8 MEG locations. There is one location per 

10 BID. The software writes the count only. The count cannot change when the queue is not 

1 1 empty. Hence there is no need for a separate byte write enable. The address is the "BID" 
_ 12 tail or head that are read from the dequeue memory, enqueue memory or the free buffer 
3 13 register. See Figures 298 and 299. 

Ml 14 The SOP, EFCI, OAM are set when the SOP cell arrives. The EOP is set in a 

J3 1 5 different BID when it arrives. There is no need to write extra cycles on EOP. 
03 16 

^ 17 Multicasting Data Structure 

3 

O 18 These are internal memories. 



fj, 19 

** 20 Multicasting FID LEAF Data Structure 

21 

22 Multicasting FID Dequeue Memory 



23 There are 1 K locations. There is one location per FID. See Figures 300 and 301 . 

24 

25 Multicasting FID Enqueue Memory 

26 There are 2 K locations. There are two locations per FID. There are 2 locations 

27 per FID. The software writes only the RED, and "NEXT_TUN". The software under 

28 normal operating traffic conditions does not need any values from this memory. See 

29 Figures 302, 303 and 304. 
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1 

2 Multicasting FID ROOT Data Structure 

3 The enqueue engine uses the next table. There are 1 K locations. The software 

4 writes the head and the tail. See Figure 305. 
5 

6 Tunneling Data Structure 

7 The memories are internal. 

8 
9 

« 10 

u 

2 12 Tunneling FID LEAF Data Structure 

p 13 Tunneling FID Dequeue Memory 

» 14 There are 1 K locations. There is one location per FID. There is one location per 

% 15 FID. The software does not write any location except the "TUNROOT". The software 

p 16 under normal operating traffic conditions does not need any values from this memory. 

0 17 See Figure 306. 
*" 18 

1 9 Tunneling FID Enqueue Memory 

20 There are 2 K locations. There are 2 locations per FID. There are 2 locations per 

21 FID. The software writes only the RED, and "NEXT_TUN". The software under normal 

22 operating traffic conditions does not need any values from this memory. See Figures 

23 307, 308 and 309. 
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1 

2 Tunneling FID ROOT Data Structure 

3 The de-queue engine uses the next table. There are 1 K locations. The software 

4 writes the head and the tail. See Figure 310. 
5 

6 Interface 

7 The interface is either internally to the memory manager, shaper or the scheduler. 

8 Else, it is to the external rams. The BYTE enable for the enqueue memory should be 

9 connected that the "BYTE_ENABLE_0" is connected to the first three bytes that contain 

10 the BID tail pointer. The second "BYTEENABLEO" should be connected to the rest of 

1 1 bytes. When writing the BID tail pointer with the value "NULL" by the dequeue engine, 
^ 12 there is no need for a read operation, instead only the first three bytes are written. The 
^ 1 3 rest of the data field is maintained as it was. The dequeue memory has no relevant data 
U3 1 4 that need to be maintained when the enqueue engine updates the BID head pointer after 
!fi 1 5 empty queue. The type field is updated with the pointer. The "TYPE" is part of the 

M 1 6 command. See Figure 3 1 1 . The signal "MEM_PFQ_FULL" is synchronized and bussed 

□ 17 through to the DBS. 

is: 18 

h& 19 CPU Interface 

H? 20 There are signals that are generated by the CPU interface and are used to control 

2 1 the memory sizes and the enable and disable signals. See Figure 312. 
22 

23 Se gmentation Interface 

24 See Figure 313. The PFQ of an egress device sends a message to the segmentation 

25 engine of Ingress devices when the number of used buffers exceeds a programmed 

26 threshold. The data will contain the port address of the egress device. The segmentation 

27 engine will multicast the message to all the input devices that instructs these devices to 

28 refrain from sending any traffic to the device of the corresponding port. See Figure 314. 

29 The message indicates the condition when the number of buffers used exceeds the 

30 threshold value by setting the flag to ' 1' . There is another message that indicates when 
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1 the condition no longer exists by setting the flag to "0". The rest of the data bits indicates 

2 the port number that is programmed by the CPU. 
3 

4 DBS & CBWFO Interface 

5 See Figure 315. 

6 DBS Interface 

7 See Figure 316. 
8 

9 Memory Manaeer Interface 

10 A command is stretched to few cycles as described below so that the number of 



1 1 pins is reduced. The interface between the 2 blocks is shown in Figures 3 1 7 and 3 1 8. 

12 The En-queue/Discard command from the memory manager to the PFQ takes 2 cycles to 
O 1 3 complete. The De-queue command from the PFQ to the memory manager takes 3 cycles 
5 14 to complete. When the BID is not valid, the "NULL" BID will be returned to the memory 

1 5 manager as the BID value. The memory manager will drop the cell whenever the BID is 

W 16 "NULL". See Figures 319 and 320. 

Lx 17 

^ 18 MEMtoPFQ 

(J! 1 9 The memory manager starts the command by asserting valid signal. The PFQ will 

ff 20 always sample the command. If it is not NOP, then the command determines the number 

B 21 of valid cycles with data. If the command is a "discard" command, then the only valid 

*~ 22 parameters are the FID; all the rest of the parameters should be ignored by the PFQ. If the 

23 command is a release then the BID is important and the rest is ignored. If it is an enqueue 

24 command, then both Dl and D2 are valid. See Figure 321. 



25 The memory manager as well as the PFQ should have FIFOS to store the 

26 commands and the parameters. 
27 

28 PFQ to MEM 

29 The command takes 3 cycles to complete. In this command the data fields are 

30 transferred in 3 cycles as shown in Figure 322. 
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1 If "REL" is set, then the memory manger can reuse the buffer, which is the BID. 

2 As a result, the memory manger cannot send a command to PFQ releasing BID that was 

3 dequeued with the "REL" bit reset. The PFQ will not check the "DEQ CNT" in the BID 

4 memory. The "DEQ CNT" is used for multicasting traffic multiple times. 
5 

6 Interface Timing between the Memory Manger and PFQ 

7 The MX1 and MX2 are not synchronous. The global synch signal is generated 

8 locally. As a result, the time slot is not adhered to in this interface. All the commands or 

9 data are stored in FIFOS after they reach the receiving device. The response to such a 

10 request will take between 16 and 24 clock cycles. For instance, the memory manger 

1 1 needs to wait between 1 6 and 24 clocks before it gets a BID after it asserts the FID 

_ 12 enqueue command. See Figure 323. 

B 

Jl 13 

"1 14 Memory Manager Interface Timing 

■S 1 5 See Figure 324. The clock is 1 00 MHZ and the commands from the PFQ to the 

: ;Z 

j t : 

Jj 16 memory manager are valid every 4 clocks. Similarly, the commands from the memory 

^ 17 manager to the PFQ are valid every 4 clocks. As a result, the PFQ block and the memory 

O 1 8 manager block will receive a command every 8 clocks of 200 MHZ. 

eP 

« 19 

^ 20 SEGMENTATION Interface Timing 

p. 21 See Figure 325. The clock is 100 MHZ and the data is serially driven on two data 

22 pins. The flag indicates if the queue is full when it is " 1 ". When the queue is full and the 

23 number of buffers used is equal or exceeds a programmable threshold, the PFQ will send 

24 a message indicating the condition by setting the flag to "1". When the number of buffers 

25 used is less than the programmed threshold by 1 0, then an equivalent message is sent but 

26 with the flag reset. 
27 

28 Software Interface 

29 The CPU interface doesn't use bursting. Data bus is still 32 bits. 
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1 

2 Features 

3 • Data bus of 32 bits. There is data bus in and out. 

4 • Single address access (not a burst access). 

5 • CPU accesses all registers in all blocks via direct read/write commands. 

6 • All external memories and most of the internal memories are accessed via indirect 

7 accesses through registers 

8 • CPU accesses all internal/external memories in all blocks via indirect accesses using 

9 the general-purpose registers dedicated for these accesses. 
10 

1 1 Address space 

12 The address space is 64 entries only. The internal registers directly mapped to 
Q 13 these addresses and external/internal memories are indirectly mapped. 

^ 14 The address spaces per lookup engine is shown in Figure 326. 
~ 15 

yj 16 Direct/Indirect Accesses 

rf 17 A direct access is used to access. If there are internal memories or registers, which 

a 1 8 are longer then 32 bits, then read or write to these locations will be executed in more then 

5 1 9 one access. For example an entry in internal memory of 50 bits width (or a register with 

2 20 the same length) will be accessed in two CPU commands first access bits [3 1 :0] and then 

0 21 [49:32]. Each register or small internal memories have their own address, which is unique 

22 to the entire chip set. 

23 An indirect access is used to access internal and external memories. Each block 

24 has a set of registers dedicated for the indirect access. One register contains the address 

25 and the command (Rd, Wr, Init etc). Few other registers contain the data for a write 

26 command. Writing to the COM/ADD register triggers the execution of the command, 

27 once the block completed the execution of the command, the COM portion of the 

28 command is cleared by the block. When a read command occurs the data is written to the 

29 registers and then the CPU reads the registers to get the data. 



163 



AZA-001/2001-P0O^P W PATENT 

1 

2 Registers 

3 See Figure 327. 
4 

5 Commands 

6 NOP. 

7 See Figure 328. 

8 Read FID Enqueue Memory (72 bits) - Up to 2M locations. 

9 See Figure 329. 

10 Write FID Enqueue Memory (72 bits) - Up to 2M locations. 

11 See Figure 330. 

12 Read FID Dequeue Memory (36 bits)- Up to 1M locations. 

13 See Figure 331. 

14 Write FID Dequeue Memory (36 bits) - Up to 1M locations. 

15 See Figure 332. 

1 6 Read BID Memory (36 bits) - Up to 16M locations. 

17 See Figure 333. 

1 8 Write BID Memory (36 bits) - Up to 16M locations. 

19 See Figure 334. 
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1 Read Stat Memory (72 bits) - Up to 2M locations. 



2 See Figure 335. 

3 Write Stat Memory (72 bits) - Up to 2M locations. 

4 See Figure 336. 

5 Setup connection command for FID 

6 See Figure 337. This command sets up a connection by accessing external 

7 memories and initializes the corresponding locations to the proper parameters. It is a 

8 write command. Clear PFQ entries for ENQ, DEQ and STAT! 



9 

10 Tear-Down connection command for FID 

1 1 See Figure 338. This command tears down a connection by accessing external 

12 memories. It is a write command. The data is all "0" 

1 3 PFQ returns all remaining BIDS on the queue of the down connection to the free 

14 link list during this command. 

15 

1 6 INIT FID MEMORIES 



17 The memories are the enqueue, dequeue, and statistics memories. See Figure 339. 

18 

19 INIT BID Memory 

20 This will build the free buffer list. See Figure 340. 
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2 Reserved Op-codes 

3 The op-codes 1 1 0 1 , 1 1 1 0 5 and 1 1 1 1 are reserved. 
4 

5 General commands 

6 The following commands are combinations of the above simple commands. A 

7 user command like this should be broken to several Read or Write commands in the 

8 driver. 
9 

10 Initialized External memories 

1 1 • Write "0" in all memory locations. 

1 2 • Write "NULL" in the enqueue memory. 

1 3 • Write "NULL" in the dequeue memory. 

14 • Build the free buffer list by linking each buffer to the next one. 



« s 15 



16 Initialized Internal memories 

1 7 • Write "NULL" in the Multicast Root memory. 

1 8 • Write "NULL" in the Multicast Leaf memory. 

1 9 • Write "NULL" in the Tunneling Root memory. 

20 • Write "NULL" in the Tunneling Leaf memory. 
21 

22 Principle Of Operation 

23 The PFQ performs the following operations: 

24 • PFQ adds a BED to the FIDS queues. In essence it enqueues a BID. 

25 • PFQ removes a BID from the FIDS queues. In essence it dequeues a BID. 
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1 • PFQ adds a BID to the free buffer queue. In essence it de allocates or returns a 

2 free BID. 

3 • PFQ removes a BED from the free buffer queue. In essence it dequeues or 

4 allocates a free BID. 

5 • PFQ increments the statistics counters. 

6 • PFQ timeouts the active FIDS. 

7 • PFQ asserts commands to the shaper and memory manager. 

8 • PFQ accepts commands from the scheduler and memory manager. 

9 • PFQ transfers statistics to the CPU. 

1 0 • PFQ executes early random discards depending on criteria that associate the FID 

1 1 with a counter and an algorithm. 
^ 12 • PFQ executes discard operations. 

=£} 1 3 • The Memory manager issues the enqueue, discard and the release. 

\$ 14 • The scheduler issues the dequeue command. 

^ 1 5 The Per flow queue will get a command from the memory manager to enqueue an 

h 16 FID. The PFQ rejects or honors the request depending on the free buffers available or 

1 7 RED. The request is honored and the enqueue operation is performed. If the request is 

§ 18 rej ected, then a discard operation is performed by making the tail of the discarded packet 

5 19 as the tail of the free buffer list. Also, the head of the packet will be linked to the previous 

U 20 free buffer tail. 

2 1 The scheduler commands are dequeue ones. The PFQ will remove the BID from the 

22 queue and transfer the information to the memory manager. 
23 

24 Error Conditions 

25 The total numbers of free buffers is compared to the number of buffers used and 

26 also to the total number of buffers on the queues. If the numbers do not match, then there 

27 will be an abort of all the operations. The PFQ will indicate the condition to the CPU. 

28 Another error condition happens when there is a dequeue request and the queue is empty. 
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1 

2 TIMING DIAGRAMS 

3 The timing diagrams are for FID and BID memory accesses. They include the 

4 accesses of the FID memory as well as the BID memory for enqueueing, dequeueing, 

5 statistics collections and timeout scanning. See Figure 341 . 

6 The memories cycles are summarized in Figures 342-346. 

7 When dequeues engine is updating the tail pointer in the enqueue memory to 

8 "NULL", there is one write operation that is performed to the enqueue memory location 

9 0. The BYTE ENABLE for the first 24 bits should be used. All the other byte enables 

1 0 should not be asserted. 

1 1 When the enqueue engine is updating the head pointer in the enqueue memory to 

12 "BID", there is one write operation that is performed to the dequeue memory. The BYTE 

1 3 ENABLE for the first 24 bits should be used. All the other byte enables should not be 



%J 14 asserted. 
2 15 

W 16 Per Flow Commands 
09 

H= 17 • The Memory manager command to PFQ: 

L 18 o ENQ {FID}: Enqueue the FLOW. 

m 19 o DSCD {FID}: Discard the FID. 

Z 20 o REL {BID}: It releases the BID. 

if 21 • The PFQ Commands to the memory manager: 

22 o SND {BED} {REL} {PORT} : Get the data cell belonging to BID and send it 

23 with port number "PORT", afterwards release the buffer if "REL" is set. 

24 o VALID {BID} : Accept the BED for the previous enqueue command. 

25 • The PFQ Commands to DBS: 

26 o SHAPE OR SCHEDULE {FED} {BED} {LOCATION} {#CELLS} {EOP} : 

27 Shape the current Flow ED FED with the location "LOCATION", EOP marker 

28 received even though should be implied. 

29 • The DBS Commands to PFQ: 

30 o DEQ {FED} : The PFQ will pass the Buffer ED to the memory manager. If 

3 1 EOP is set, then send a shape command to shaper. 

168 



AZA-001/2001-P00 



PATENT 



1 • The PCI Commands to PFQ: 

2 o Get [ADR] {CNTS} : Get the counts values from the statistics memory. 
3 

4 



"*4 

.35. 

m 
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1 Data Base 

2 The data base block is responsible for managing the packets link list per FlowID 

3 for FlowIDs in the Shaper and FlowIDs in the scheduler. The block receives an En-queue 

4 command from the PFQ and sends the FlowID either to the Shaper or to the Scheduler. A 

5 De-queue command starts with a port selection based on a port calendar and then a 

6 FlowID is selected between the scheduler and the shaper. 

7 Shaper traffic has priority over scheduler traffic. Shaper traffic is divided to 8 

8 classes of strict priority, once all shaper' s classes are empty for a given port, the 

9 scheduler is allowed to schedule. 
10 

11 Features 

12 1. This block operates at 200MHz clock. 

13 2. Support up to 1M FlowIDs. 

14 3. Support a link list of packets/cells up to 8M cells. 

1 5 4. Performs the following in one port slot (8 cycles): 

1 g a . One input phase from the PFQ and redirect to an input phase in the Shaper 

17 or in the Scheduler or to both (when traffic is just metered) 

18 b. One output phase of the Shaper. 

19 c. One output phase of the Scheduler. 

20 5. Support 8 strict priority classes for the shaped traffic per port. 

21 6. Performs port scheduling for the shaped traffic base of strict priority within the 

22 block. 
23 

24 Interface 

25 A global block diagram of the database block is shown in Figure 347. 

26 Figure 348 shows the interface to/from the data base block. See also Figure 349. 
27 

28 Functional Description 

29 The DBS block acts as the holder of the information per FlowID and the packets 

30 within the FlowID for the Shaper and the Scheduler blocks. The DBS block can be 

3 1 divided to several processes during one port slot: 
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1 

2 PFQ Input Phase 

3 For a packet traffic the PFQ first assembles a complete packet and then transfer 

4 the packet to the DBS block, when in a cells traffic, a cell is being sent to the DBS each 

5 time it arrives. One cell in cells traffic is considered as a packet with a length of one cell. 

6 When a packet arrives to the DBS block from the PFQ, the DBS first links the packet to 

7 the FlowID and also generates an input phase to the Shaper or to the Scheduler or to both 

8 based on a parameter bit per FlowID. 
9 

10 Shaper Output Phase 

1 1 The output phase of the Shaper is being performed on a cell basis, means each cell 

12 of a packet generates an output phase from the Shaper. Once a cell is driven from the 

1 3 Shaper, the DBS block decrement the cell counter for the specific packet. If it' s the last 

14 cell of the packet, the DBS links the packet to the shaper's output memory per port. The 

15 DBS also generates an empty indication to the Shaper if there are no more packets to 

16 shape for the FlowID. 
17 

18 Port Calendar 

1 9 A port calendar is a free running port selection based on a memory programmed 

20 by the user. Once a port is selected for the current output phase, the port number is 

21 transferred to both scheduler and shaper's output memory. Both blocks select their output 

22 FlowID for the output stage. If the shaper's output memory has a valid FlowID and the 

23 scheduler is not in the middle of a packet, then the FlowID to be driven to the PFQ is 

24 always from the shaper's output memory. Shaped traffic has priority over un-shaped 

25 traffic. 

26 Each output phase that starts with the port calendar also starts with checking the 

27 port status. Only if the port calendar has a valid port and that port is not full according to 

28 the port status, a valid port is driven to the scheduler and to the shaper. 
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1 

2 Port Status 

3 This block receives the status information from the output SPI interface for 64 

4 ports plus one CPU port. See the timing diagram of that interface below. 
5 

6 Scheduler Output Phase 

7 Once a port is selected, the scheduler performs it's own output phase. The DB 

8 block needs to support this phase the same way it supports the shaper's output phase. 
9 





1 n 


Rlnrk Tipscrintion 




1 1 
1 1 


The following are the algorithms to perform in each of the previous phases: 




12 








PFO Ontnut Phase 






1 The PFO drives the following signals: VALID, FID, NCELL, BID, CLPI 


: J : 


15 


2. Read FID_mem[FID] {rp, wp, e, celljmt, ncell, dp, port, qos, c_p t 


y 


16 


CPU jport, shape} 


= 


1 / 


7 yj xvn < = RID 


=*=sr 


I o 


4 n e < = 0 


CP 


1 A 

19 


J. if(e — I) 




20 


1 ) if(shape_meter==0 \ shape==l) perform input phase to Shaper: 




21 


VALID, FID, qos, ncell 




22 


2) if(shape_meter==0 | shape==0) Perform input phase to Scheduler: 




23 


VALID, FID, qos, CPU _port 




24 


3) n_rp<=BID 




25 


4) n_ncell<= NCELL 




26 


5) n_cell_cnt <= NCELL 




27 


6) n_clp<=CLPI 




28 


7) pkt_next_we = 0 




29 


6. else 




30 


I ) if (shape _meter==0 ) perform input phase to Shaper: 




31 


VALID, FID, qos, ncell 
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2) njrp <= rp 

3) njxcell <= ncell 

4) njcelljmt <=cell_cnt 

5) n clp <== dp 

6) pktjiextwe <= 1 

7. if(VALID==l) Write FID_mem[FID]<= 

{n_rp, n_wp, n_e, njicell, n_cell_cnt, n_clp, qos, cj>, CPUjport, shape} 

8. if(VALID==l & pkt_next_we==l) Write pktjnem[wp] <= 

{ncell dp, FID, next, [dbs_dpkt_word_en[l:0] =11] <= {NCELL, CLPI, 
0, BID} 



Shaper Output Phase 

1. The SHP drives the following signals: VALID, FID, VIOLATOR 

2. Read FIDmemfFIDJ {rp, wp, e t cell_cnt, njzell, dp, port, qos, cj?, 
CPU _port, shape} 

3. Wait until read is complete 

4. cur eop < = cell cnt ==1 

5. curjempty <== curjeop & rp==wp 

6. Read pkt_mem[rp] {ncell, dp, next, FID} 

7. Read shpjout j>ort[port] {p_rp, p_wp, pjz} 

8. if(curj>op==l) n_rp<= next; njzellcnt <= ncell; nji^cell <= ncell; 

9. else n_rp<= rp; n_cell_cnt <= celljcnt -1; njicell < = njtell; 

10. if(cur_empty ) n_e <= 1 

11. n_FID<=FID 

12. n_clp <= enjdp_change ? (VIOLATOR \ dp) : dp 

13. if(cur_eop) 

shpjout jportjwe <= 1 
n _p_e < = 0 
n __pjwp <- rp 
if(P^e)n_p_rp<=rp 

14. else 
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shp out jportwe <= 0 

15. send cur empty indication to Shaper 

16. if(VALID==l) Write FID_mem[FID] <= 

{n_rp, wp, n_e, n_cell_cnt, n_n_cell, qos, c_p, CPU_port, shape} 

17. if(VALID==l & curjeop) 

Write shp_out _port[port] <= {n _p_rp, n_p_wp, n_p_e } 

18. if(VALID==l & cur_eop & p_e==0) 

Write pkt_mem[p_wp][next][dbs_dpkt_work_en[l] =1] <= {rpj 

19. if(VALID==l & cureop) 

Write pkt_mem[rp][dbs_dpkt_work_enfO]=l] <= { n_clp, n_cell, n_FID} 

DBS Output Phase (Combined with Scheduler Output Phase) 

1. Read portjcalendar mem[count] {portid, valid, jump} 

2. if(jump==l) count <= 0 

3. else count = count + 1 

4. send shaper and scheduler: portid, valid 

5. wait for 4 cycles to receive SHP_FID, SHP_VALID, SCH_FID, SCH VALID, 
SCH_MID 

6. get PFQ FULL indication from PFQ block 

7. if(valid & -PFQ FULL & SHP_VALID & (~SCH_MID \ ~SCH_VALID)) 

1) out Jromjshp <= 1 

2) out Jromjsch < = 0 

if(valid &-PFQJFULL & SCH_VALID & (~SHP VALID \ SCHJfID) 

1) out Jrom shp <= 0 

2) out _from_sch <= 1 

8. iffout Jrom_shp==l) 

1 ) Read shp _priority_mem[portid] { empty} 

2) Select class per portid based on empty : class _addr = {portid, class} 

3) Read shp_out_mem[class_addr] {p_rp, p_wr, p_e} 

4) Read pkt_mem[p_rp] {ncell, dp, next, FID} 

5) Send to PFQ: FID, portid, p_clp, valid(=l) 
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1 6) If(ncell==l) 

2 a. n _p_rp <= next 

3 b. if(p_rp==p_wp) n_p_e <= 1 

4 c. njicell <= 0 

5 7) else 

6 a n _p_rp <=p_rp 
1 b. n _p_e <=p_e 

8 c. njicell <= ncell - 1 

9 8) Write shp_out_mem[class_addr] <= {n_p_rp, p_wp, n_p_e} 

1 0 9) If(n_p_e== 1 ) Write shp _priority[portid] <= new empty indication 

11 10) Write pkt_mem[p_rp] < = {n ncell} 

12 9. if(out Jrom_sch==l) 

Q 13 1) Read FID_mem[SCH_FID] 

S 14 {rp, wp, e, cell_cnt, n_cell, port, qos, c_p, CPU_port, shape} 

"J 15 2) eop <= (cel_cnt==l) 

Q 16 3) empty <= (cell_cnt==l & rp==wp) 

2 17 4) busy <= PFQ_FULL 

* 18 5) Read pkt_mem[rp]{ncell, dp, next.FID} 

I 19 6) Send eop, busy, empty to scheduler 

g 20 7) if (eop) n_rp<= next; n_cell_cnt <= ncell; n_n_cell<= ncell; 

Q 21 8) elsen_cell_cnt<=cell_cnt-l;n_n_cell<=n_cell; n_rp <= rp; 

22 9) if(empty) n_e <= 1 

23 10) Write FID_mem[SCH_FID] <= 

24 {n_rp, wp, n_e, n_cell_cnt, n_cell, port, qos, c_p, CPU_port, shape} 



25 

26 Port Status 

27 This interface drives the status of the 64 ports in the output SPI and the CPU port. 

28 It is a continuous interface. Every 8 cycles of 200MHz the status indication of the 65 

29 ports is updated to a 65 bits register. The first cycle of the eight is marked with a specific 

30 signal (sync signal). Figure 350 shows the interface. 
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1 The database block needs to synchronize the incoming status bits to the local 

2 200MHz clock and then use the status bits. A synchronization FIFO is being used, but 

3 since the write and read frequencies are the same, there is no empty or full indications. 

4 Reads will always lag 3 cycles after the writes. To achieve that, the read pointer is always 

5 the value of the write pointer minus 3 . Also the first 3 reads after reset should be ignored. 
6 

7 Full indication from PFQ 

8 A full indication from the PFQ can be asserted any time. Once a full indication is 

9 asserted, the DBS first complete the current output phase and then freeze the increments 

10 of the port calendar. From that moment until this indication is de-asserted no output 

1 1 phase is performed from the scheduler or from the shaper's output FIFOs. The shaper's 

1 2 output phase to the DBS should still continue without any connection to the full 

13 indication. 
14 



2 15 CLP Bit 



W 1 6 The CLP bit is modified in the Database and then is. sent back to the PFQ during 

1 7 the output stage. Only FlowIDs that are shaped can have their CLP bit changed, FlowIDs 

1 8 to the scheduler have the CLP bit unchanged. The violation indication from the shaper 

19 should set the CLP bit to 1 , if no violation indication arrives then the CLP bit is to remain 

20 the same as it was. 
21 

22 Memory Description 

23 

24 Chip-Select indication for the PKT Memory 

25 The packet memory can support up to 8M locations (the address bus has 23 bits). 

26 Since there are no SSRAM memories in the market that have that amount of entries, the 

27 DBS provides chip-select signals so that several memories can be connected to get up to 

28 8M locations. 

29 The chip-select signals are used only when supporting 2M locations and up. The 

30 generation of the chip-select depends on the input from the CPU interface called 

3 1 CPU_CS_SEL[1 :0] as follow: 
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1 00 - generate chip-select based on address[20: 1 9] 

2 01 - generate chip-select based on address[2 1 :20] 

3 10 - generate chip-select based on address[22:21] 

4 1 1 - Reserved 

5 for example, if supporting 2M locations only and the memory devices used have a depth 

6 of 5 1 2K locations, then CPU_CS_SEL bus should have the value of 00. the 

7 chip_select[3 :0] will have the following meaning: 

8 chip_select_l[0] - selects memory block for addresses 0 - 1/2M 

9 chip_select_l[ 1 ] - selects memory block for addresses 1 /2M - 1 M 

1 0 chip_select_l[2] - selects memory block for addresses 1M - 1 1/2M 

1 1 chip_select_l[3] - selects memory block for addresses 1 1/2M - 2M 

12 In this case address bus bits [18:0] are going to all memory blocks (Note, each block has 

1 3 only 5 12K location, which translated to 19 bits), the chip_select_l bus controls 4 memory 

14 blocks, therefore, the total locations supported is 2M. 
15 

16 DBS FlowID Memory 

17 

18 FlowID Memory 

19 Address: FID, Length: 1M, #bits: 91 

20 See Figure 3 5 1 .The user has to initialize the above fields, which are marked as 

21 software using setup connection command. 

22 

23 PKT Memory 

24 Address: BID, length: 8M, #bits: 55 

25 See Figure 352. 
26 



177 



AZA-001/2001-P00 




PATENT 



1 Port calendar memory 

2 Address: counter, length: 192, #bits: 8 

3 See Figure 353. The user should initiate this memory prior to any traffic based on the 

4 users input. 



5 

6 Shpstrict memory 



7 Address: port, length:64, #bits: 12 

8 See Figure 354. 
9 

10 Shp_out_port memory 

1 1 Address: port, length: 5 1 2, #bits: 49 

12 See Figure 355 

13 . 

14 Registers Description 

15 See Figure 356. 



16 

17 CPU Commands 

18' 

19 Read external FID memory 

20 See Figure 357. 

21 A[0] - C_P (bit [6] in the register) 

22 A[l] - CPUPORT (bit [7] in the register) 

23 Pri[2 : 0] - Priority for shaped traffic 

24 S - Shape (bit [23] in the register) 

25 C[0] - dpi (bit [ 1 0] in the register) 

26 C[l] - Empty (bit [1 1] in the register) 
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1 

2 Write external FID memory 

3 See Figure 358. 

4 A[0] - C_P (bit [6] in the register) 

5 A[ 1 ] - CPU.PORT (bit [6] in the register) 

6 Pri[2:0]- Priority for shaped traffic 

7 S - Shape (bit [23] in the register) 

8 C[0] - dpi (bit [ 1 0] in the register) 

9 C[ 1 ] - Empty (bit [ 1 1 ] in the register) 

10 

1 1 Setup FID connection 

12 See Figure 359. 

13 A[0] - C_P (bit [6] in the register) 

14 A[l] - CPU.PORT (bit [6] in the register) 

1 5 Pri[2:0] - Priority for shaped traffic 

16 S - Shape (bit [23] in the register) 

17 During the setup connection command, the user defines the above fields only. 

1 8 The fields RP, WP, NCELL, CELL CNT and CLP should be reset to 0 by the hardware 

19 The field EMPTY should be set to 1 by the hardware. 

20 

21 Read external PKT memory 

22 See Figure 360. 

23 L[0]-CLP 

24 L[l] -Reserved 
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2 Write external PKT memory 

3 See Figure 361. 

4 L[0]-CLP 

5 L[l] - Reserved 

6 

7 Read port calendar memory 

8 See Figure 362. 

9 Data[5:0]-portid 

10 Data[6] - valid 

11 Data[7]-jump 

12 

1 3 Write port calendar memory 

14 See Figure 363. 

15 Data[5:0]-portid 

16 Data[6] - valid 

17 Data[7]-jump 

18 

19 Read SHP Strict memory 

20 See Figure 364. 

21 P[2:0] - previous QOS pointer 

22 P[3] - previous QOS pointer valid 
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1 

2 Write SHP Strict memory 

3 See Figure 365. 
4 

5 Read SHP out memory 

6 See Figure 366. 
7 

8 Write SHP out memory 

9 See Figure 367. 
10 

1 1 Init SHP_out_port memory 

12 See Figure 368. This command generates write accesses to all entries in the 

13 Shp_out memory (64 entries) with a specific data of: p_rp = 0, p_wp = 0, p_e - 1 



14 

15 Init Shp_strict memory 

16 See Figure 369. This command generates write accesses to all entries in the 

17 Shpstrict memory (64 entries) with a specific data of: empty = 8'bl 1 1 1_1 1 1 1, 

1 8 prevqosv - 0, prev qos = 0 
19 

20 Reserved Opcodes 

2 1 The opcodes: 1 1 1 0, 1 1 1 1 are reserved. 
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Timing Diagram 

Output phase to PFQ from SHP output memory. See Figure 370. 

Notes for Figure 370: 
PFQ Output, SHP Output, SCH Output DBS Output 

Notes : 

1 .Slot 4 has data dependency on slot 6 if SHP FID and PFQ FID are same. Use following 
algorithm for this data dependency 

if (shp_FID = PFQ_FID & shp_out_valid & PFQ_in_valid( 
@slot 6 empty <= 0 

wp<=BID(PFQ) 

if (shp_empty) rp <= BID(PFQ) 

else rp <= n rp 

2. Slot 4 has data dependency on slot 7 if scheduler FID is same PFQ FID. In that case 
if (schFID = PFQJFID && sch_out_valid & PFQ_in_valid) 
@slot 7 

empty <= 0; 

wp <= BID(PFQ) 

if (sch_empty) rp <= BID(PFQ) 

else rp <= n_rp; 

There are no data dependencies on the pkt memory since a packet can exist only in one 
place at a given time. 



Interfaces 
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1 Shaper 



2 See Figure 371. 
3 

4 Scheduler 

5 See Figure 372. 
6 

7 PFQ 

8 See Figure 373. 
9 

10 Output Scheduler 

1 1 The output scheduler receives unshaped flow IDs from the Data Base Module(it is 

12 referred as DBS) and links them in its database. Then it chooses a flow ID based on its 

13 scheduling algorithm and outputs the flow ID to the PFQ engine. Note that unshaped 

14 flow IDs will get available bandwidth which is not used by shaped FLOW IDs. Shaped 

1 5 flow IDs are scheduled for output by the DBS and these flow IDs get committed 

16 bandwidth. In other words Shaped flow IDs from the DBS always get priority over 

17 unshaped flow IDs from scheduler unless it is middle of packet. When DBS received a 

1 8 flow ID, flow ID is either sent to shaper(if it to be shaped) or scheduler(if it is not to be 

19 shaped). Once flow ID is received from DBS, scheduler links this flow ID based on the 

20 port number and QOS. Ports are always serviced in round robin fashion by the database. 

21 DBS informs the scheduler the port number which will be served in this time slot( 8 

22 clocks). If there are one or more flow IDs pending for this port then a flow ID is selected 

23 for scheduling based on the following criteria. 

24 The scheduler implements 2 levels of scheduling 

25 • Level 2: Weighted-Round-Robin algorithm combined with strict priorities and 

26 best effort to schedule QOSs. 

27 • Level 3 : Round Robin between FlowIDs 
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1 



The scheduler also interfaces with a CPU through the software interface. The CPU 

2 can read or write the scheduler internal registers and all the memories. Furthermore, the 

3 scheduler receives flow control status from the re-assembly engine, the PFQ engine and 

4 the egress chip. 
5 

6 Features 

7 1. The scheduler operates at 200MHz clock. 

8 2. Round-robin algorithm between Flows. 

9 3. Combination of weighted round-robin algorithm and strict priority for QOSs. 

10 4. Support up to 1048576 Flows (1M). 

1 1 5. Support 192 port of STS1 rate each up to one OC192 port. CPU port is always OC3. 

1 2 6. Supports up to 1024 QOSs that are shared between the output ports. 

? 13 • Up to 8 QOSs per port. CPU port has only one QOS, which is not accounted in 

S| 14 the IK. 

J 15 • The 8 lh QOS per each port is acting as a best effort QOS, means this QOS will be 

W 1 6 selected only if all other QOSs are empty. 
03 

u 17 7. Two modes of operations: 

U 18 • Egress Mode: (Normal operation) Each port has up to 8 QOSs, there are 64 

5 1 9 output-ports plus 1 CPU port which has only one QOS. 

S 20 • Ingress Mode: Support only one output port. All QOSs belong to that one port. 

2 2 1 Virtually there are still up to 256 ports with up to 8 QOSs dedicated to each. A 

22 separate port status information bits will arrive from the Egress chip to block the 

23 QOSs in the Ingress chip, which send traffic to the congested ports in the Egress 

24 chip. This mode of operation affects the port calendar only. 

25 Figure 374 shows both applications with respect to the switch: 

26 8. Status for Ingress chip applications comes from the Egress chip. 

27 9. External memory supported is pipelined ZBT memory. Micron provides pipelined 

28 ZBT memories of 512K x 32/36 or 1M x 18. 

29 1 0. In case of scheduling packets, only one packet can exist in the scheduler for a specific 

30 FlowID. Once this packet is scheduled the database block (DBS) will issue another 



184 



AZA-001/2001-P0! 




PATENT 



1 packet of the same FlowID to the scheduler if there is one linked in the DBS. In case 

2 of scheduling ATM cells, each cell is treated as a one cell packet. 

3 11. Accept to link a flow from the DBS once every 8 cycles of 200MHz. 

4 12. Since 8 clock cycles are needed to service a port (a cell), a total of 520 (64 ports + 

5 CPU port) cycles are needed to keep up the lOGbs traffic rate. 
6 

7 Interface 

8 Figure 375 shows the interface to/from the scheduler block. Figure 376 shows the 

9 detailed interface to other blocks. Signal Descriptions are found in Figure 377. 
10 

11 Functional description 

1 2 Flow ED is given to scheduler by DBS if flow ID given by PFQ is not to be 

O 1 3 shaped. Based on a port and a QOS a flow ID belongs to, the scheduler informs DBS to 

\j 14 link up the flow ID in the DBS's linked list memory structure. DBS selects port to serve 

"12 1 5 and inform scheduler port number which will be served in the given slot. Scheduler then 

E 1 6 selects a QOS based on the strict priorirty or WRR algorithm among all the QOSs of the 

H 1 7 port, and finally chooses a flow ID based on the round robin algorithm between all the 

L 1 8 flows of the QOS . The scheduler sends the scheduled flow ID to DB S for scheduling. At 

P 1 9 this point DBS decides whether flow ID from scheduler will be sent out or flow ID from 

U 20 shaper will be sent out. Always if there is flow ID from shaped traffic for this port, it will 

Q 21 be sent out first. In that case DBS informs scheduler not to update any parameters as flow 

22 id from scheduler has not been scheduled. When there is no flow ID from shaped traffic 

23 for this port, then flow ID from scheduler will be sent out. In that case, scheduler updates 

24 all parameters. Also if the current cell scheduled is the last cell (in case of ATM traffic, 

25 Every cell sent out will be marked as eop ) then End Of Packet(eop) signal is sent from 

26 DBS to scheduler. Also if the current packet is the last packet linked for this flow ID, 

27 empty will be asserted by DBS to scheduler. If it eop and not empty then scheduler has to 

28 rotate flow IDs. Scheduler will link current read pointer(f_rp) to end of link(f wp). This 

29 information is sent to DBS. Also next_FID given by DBS will become head of link list 

30 for this QOS. 
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The following explains all the scheduling algorithms employed by the scheduler 

2 and the scheduler' s linked list memory structure. 
3 

4 Round Robin Algorithm 

5 All the elements to be scheduled are linked together in one link list. There are two 

6 pointers to that link list, a head pointer and a tail pointer. A selection is basically 

7 forwarding the head pointer. There are 2 options to continue the algorithm after selecting 

8 the element: the element is linked to the end of the list and become the tail pointer while 

9 the next element becomes the head of the list. The second option is that the element is 

1 0 removed from the link list, in this case only the pointer is changed to the next element in 

11 the list. See Figure 378. 

1 2 A Round Robin algorithm is used for the FlowIDs. The port calendar is 

1 3 implementing the same but without removing or adding elements to the link list, means a 

14 fixed list. 
15 

16 Weighted-Round-Robin Algorithm 

1 7 This algorithm is an extension to the Round Robin one. Each element in the link 

1 8 list has a weight and will be selected as much as the weight allows. This algorithm should 

1 9 generate a relative bandwidth to each element according to the total elements linked: 

20 BW k = Weight k /XWeighti 

2 1 To implement this algorithm there is a need for two link lists. Active link list, which 

22 holds the active elements to schedule, once an element k was schedule weight k times, it is 

23 moved to the waiting list. Once all the elements in the active list exhausted their weight 

24 and moved to the waiting list (active list becomes empty) the waiting list becomes the 

25 active list and the above formula is performed on the elements in the new active list. 

26 During processing of the active list, if a new element arrives, it is linked to the waiting 

27 list and will participate the scheduling only after the current cycle of the above formula is 

28 done. Each time an element is moved to the waiting list, the weight count will be set to 

29 the original weight. See the example in Figure 379. This scheduling scheme is used to 

30 schedule the QOSs per port. 



31 
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1 Smoothing the bursts 

2 The WRR algorithm can cause long bursts of cells for the same FlowDD/QOS if 

3 the weight is high. In order to make smaller bursts of data for the same FlowID/QOS, the 

4 QOSs are rotating in the active list several times before exhausting their weight. A Factor 

5 parameter is introduced per port, the weight of all QOSs is divided by the Factor and the 

6 result is the number of times they will be scheduled before moving to the end of the 

7 active list. For an example see Figure 380. 
8 

9 Negative Weight 

10 When scheduling packets, the QOS of the specific FlowID will be scheduled until 

11 an EOP indication arrives, it is possible that the packet is bigger then the weight of the 

12 QOS so that the count can become negative. Once EOF arrives if the weight count is still 

13 positive, the QOS can be scheduled for another packet. If the weight count is negative, 

14 the QOS is linked to the waiting list. The weight of the QOS is added to the weight count 

15 as well. Once he waiting list becomes active and the QOS is to be scheduled, if the 

16 weight count is still negative weight will be added again and the QOS will be sent to the 

1 7 waiting list again without scheduling. 

18 This is similar to a negative credit, if a QOS consumes all the credits and beyond 

19 it has to accumulate enough credits in order to start scheduling again. 

20 Best effort QOS 

21 This is another flavor for the selection of the QOS per port. A dedicated QOS 

22 (QOS7) is called the best effort QOS. It does not participate in the strict priority scheme 

23 or in the weighted-round-robin scheme. This QOS is scheduled only if all QOSs that 

24 belong to the strict priority are empty and all the QOSs that belong to the WRR are empty 

25 as well. A simple way to describe it is that this QOS is scheduled only if all other QOSs 

26 are empty. 
27 

28 Strict Priority 

29 Select a QOS can be done either by WRR algorithm or by assigning priorities to 

30 QOSs. Each port can assign priorities and WRR QOSs as shown in Figure 381 . 
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2 Selecting QOS 

3 As shown in Figure 382 there will be two QOS queues maintained per port. One 

4 is active list and other is waiting list. When a flow ID arrives from DBS, it will always be 

5 put in waiting list. Since there are eight QOSs per port each of this list is 8-bit wide and 

6 they are called QA_EMPTY(active list) and QW_EMPTY(waiting list). When 

7 corresponding bit is 1 then this QOS is empty and nothing to schedule. When 

8 QA_EMPTY, is all ls(no QOS to schedule) QW_EMPTY is copied to QAEMPTY, and 

9 corresponding bits QW EMPTY are made 1 . Copying is done only for round robin 

1 0 priority QOSs. Strict priority QOSs will always remain in waiting list. At any point, 

1 1 when the port is to be serviced, based on the priority allocation for that port, first checked 

12 to find out if there are any strict priority QOSs are not empty in waiting list. If so then 

1 3 these QOSs are served first. Only when all strict priority QOSs are empty, then round 

1 4 robin QOSs are selected from active list. At any time if eop is sampled for the current 

1 5 QOS, this QOS will be served till eop irrespective of priority. Following algorithm 

16 explains how to select QOS for servicing once port is selected. 

1 7 if (prev_qos_v = valid) then current_qos = prev_qos 

1 8 else if (priority [0] & ~qw_empty[0]) current_qos = 0 

1 9 else if (priority [ 1 ] & ~qw_empty[ 1 ]) current_qos = 1 

20 else if (priority [2] & ~qw_empty[2]) current_qos = 2 

2 1 else if (priority[3] & ~qw_empty[3]) current_qos = 3 

22 else if (priority[4] & ~qw_empty[4]) current_qos = 4 

23 else if (priority[5] & ~qw_empty[5]) current_qos = 5 

24 else if (priority[6] & ~qw_empty[6]) current_qos = 6 

25 else if (priority[7] & ~qw_empty[7]) current_qos = 7 

26 elseif(weight_mf !=0 & qa_empty[active_ptr] = 0) 

27 currentqos = active_ptr 

28 elseif(&qa_empty[7:0]=0) 

29 // repeat these steps for 0 to 7 

30 case (prev_qos) // synopsys parallel_case 

31 000: 
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1 case(n)O) 

2 qa_empty[l] : currentqos = 1 

3 qa_empty[2] : current_qos = 2 

4 qa_empty[3] : current qos = 3 

5 qa_empty[4] : current_qos = 4 

6 qa_empty[5] : current_qos = 5 

7 qa_empty[6] : current_qos = 6 

8 qa_empty[7] : current qos = 7 

9 qa_empty[0] : current qos = 0 
10 

1 1 To load pending QOSs from waiting to active. This function is performed any 

12 time active list is empty. Note that only non_strict priority QOSs are loaded to active list 

1 3 and strict priority QOSs are always serviced from waiting list. To load non priority QOSs 

14 to active list use 

15 if (&qa_empty[7:0] = 1) 

16 qa_empty[0] = priority [0] | (~priority[0] & qw_empty) 

1 7 qw_empty[0] = ~prioirty[0] | (priority[0] & qw_empty [0]); 
18 

19 Levels of scheduling 

20 Once a port is selected according to the port calendar, a QOS is selected between 

21 all the QOSs that belong to the port. Once a QOS is selected a FlowID is selected 

22 between the FlowIDs that belong to that QOS. See Figure 382. 
23 

24 Weight Quota 

25 The weight of a QOS is calculated based on two values. The first value is the 

26 weight parameter, which is stored per QOS in the QOS descriptor memory and the 

27 second parameter is the weight_quota, which is a programmable vale per all the QOSs. 

28 The weight of a QOS is the multiplication of the weight and the weight_quota. 
29 

30 Input Algorithm 

3 1 1. Get FlowID and QOS from the shaper/DBS. 
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1 


2. Read QOS memory: 




2 


, F _ WP = QoS_mem[F_QOS][F_WP] 




3 


• FJEMPTY = QoS_mem[F_QOS][F_EMPTY] 




4 


. Q_NUM = QoS_mem[F_QOS][Q_NUM] 




5 


. PORTJD = QoS_mem[F_QOS][PORT_ID] 




6 


3. Read Port memory: 




7 


. Q/\ EMPTY, QWJEMPTY = port_mem[PortJD)[ QAJEMPTY, QW_EMPTY] 




8 


• PRIORITY = port_mem[Port_ID][PRIORITY] 




9 


4. Write to memories: 




10 


//Link FlowlD to previous one 




11 


• Write flow_mem[F_WP][F_NEXT] <= FlowlD 




12 


II Update tail of the link list of the FlowlDs 




13 


. Write QoS_mem[F_QOS][F_EMPTY] <= 0 




14 


. Write QoS_mem[F_QOS][F_WP] <= FlowlD 




15 


. If (FJEMPTY == 1 ) Write QoS_mem[F_QOS][F_RP] <= FlowlD 




16 


II Mark the QOS not empty in the port if needed 




17 


lf(QA_EMPTY[QOS_NUM]==1)&(QW_EMPTY[QOS_NUM] == 1) 


w 


18 


. Write port_mem[PORT_ID][QW_EMPTY[QOS_NUM]] <= 0 


06 


19 


II Link port to calendar in case of Ingress chip configuration 


3 


20 


If ((MODE— 1) & ( \QA_EMPTY) & ( \QW_EMPTY)) begin 


yi 


21 


Write port_mem[INGRESS_PTR[15:8]] <= PORTJD 




22 


INGRESS_PTR[15:8] <= PORTJD 


TEES 
1 5 


23 


lf(INGRESS_PTR[16]==0) INGRESS_PTR[7.0] <- PORTJD 


M= 


24 


INGRES_PTR[16] = 0 


26 


end 




AT 

LI 
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28 
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29 


options in step 9. 




30 


1. Get PortlD from Port Calendar. 




31 


2. Read from port memory. 




32 


• All parameters 




33 


3. If fl (QOS_EMPTY)) == Oj terminate output stage 




34 


Else continue to step 4 




35 


4. If active list is empty -> Move waiting list to Active list and clear waiting list. 




36 


5. Select QOS according to priority, WRR and previous QOS in middle of packet indication. 
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6. Read from QOS memory, address is {Portlu, uuty. 




2 


• All parameters 




3 


7. Send to PFQ the FJRP value (This is the scheduled FlowlD) 




4 


8. Read FlowlD memory 




5 


• All Parameters 




6 


9. Get port status from Status block. 




7 


10. Select one of the following options depending on parameters from all memories: 




8 


o deiinkjlowid 




9 


o movejiowidjojandjofjactive 




10 


o dont_move_flowid_and_update_cell_cnt 




11 


o do_nothing_to_flowid 




12 


o delink_QoS 




13 


o move_QoSJo__end_of_active 




14 


o don t_move_QoS_and_update_weight_m 




15 


o move_QoSJo_waiting 




16 


o do_nothing_to_QoS 


iH 


17 


o For Ingress chip only: 


TOT 

yj 


18 


■ move jport_to_end_of_link_list 


HI 


1 Q 


■ Hf^fink nnrt 


M 


20 




3 


21 


1 1. Update internal and external memories according to above decision. 


?=? ' : 
L— is 


22 




ires: 


23 


Algorithm to calculate q_weight_m during output phase 


: a 
i 


24 


if (qosjzmpty) 




25 


q_weight_m <= 25 'h0 




26 


else if(qjveightjn<$ & eop) // move to waiting list 




27 


q weight _m <= q_w eight _m - 1 + (q_weight * weight jjuota) 




28 


else if(q weight m!=25 'hi 000000) // not the maximum negative value 




29 


q_w eight _m <= q_weightjn - 7 




30 






31 


Memory links for Input Phase 




32 


The input phase starts with a FlowlD from the DBS. With this parameter all memories 




33 


are accessed in sequence to get all relevant parameters required for the input phase. See 




34 


Figure 383. 
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1 

2 Memory links for Output Phase 

3 The output phase starts with a port assigned by the port calendar inside the DBS 

4 block. The port parameters are read from the internal memory, and a QOS is selected for 

5 that port based on the none-empty QOSs and the strict priority. The address to the QOS 

6 memory is {PortID, QOSSELECTED} . The read pointer in the QOS parameters is the 

7 FlowID to send to the PFQ. Once reading the parameters of that FlowID it can be sent to 

8 the PFQ together with the EOP indication. See Figures 384 and 385. 
9 

10 CPU Port 

1 1 CPU port is different then the rest of the 64 ports supported. The CPU port does 

12 not have any QOS and it's rate is fixed to OC3. FlowIDs are linked to the CPU ports 

1 3 using the external NEXT memory as the next field. FlowIDs are scheduled based on a 

1 4 simple round robin between them. 
15 

16 Blocks Description 

17 The scheduler is divided to several different functional blocks as follow: 
18 

19 Input Phase Block 

20 The input phase contains a small FIFO to accumulate the link requests from the 

2 1 shaper. These link requests are served once an input slot is available. During this stage a 

22 FlowID is linked in a link list and if needed a QOS is linked to the waiting lists. Also the 

23 port associated with that FlowID is marked as non-empty. 
24 

25 Output Phase Block 

26 The output phase block receives a port from the port calendar. Based on the status 

27 of that port, a decision is made if to go ahead with scheduling (if the port is not full) or to 

28 dedicate the bandwidth to the software (if the port is full). If scheduling is ok, a QOS is 

29 selected according to a weighted-round-robin algorithm and based on that QOS a FlowID 

30 is selected. This FlowID is then transferred to the PFQ. 
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1 If the cell count of a FlowID becomes zero, means a full packet was sent, the 

2 FlowID would be de-linked from the link list if there is no other packets linked to this 

3 FlowID. It is possible that because of that the QOS will be de-linked as well. 
4 

5 Software Interface 

6 Software needs to access the scheduler database for several purposes: 

7 1 . Initialization after reset: port calendar and QOS databases as well as internal registers. 

8 2. When setting-up a connection, to write the FlowID parameters. 

9 3. For testing purposes of the internal memories mainly. 

1 0 All software accesses to the scheduler are done through specific registers during the 

1 1 dedicated time slots. 
12 

13 Memory Description 

1 4 The following is a description of the memories needed to implement the 

15 scheduling algorithm. 
16 

1 7 External Memory - Next FlowID 

1 8 Address: Flow ID, Length: 1M (20 bits wide) 

19 See Figure 386. 
20 

2 1 Internal memory - QOS Parameters 

22 Address: QOS, Length: IK (1 7 bits wide) 

23 See Figure 387. 
24 

25 Internal memory - QOS Descriptors 

26 Address: QOS, Length: IK (66 bits wide) 

27 See Figure 388. 
28 

29 Internal memory - Port Parameters 

30 Address: Port ID, Length: 64 (2 1 bits wide) 

31 See Figure 389. 
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1 

2 Internal memory - Port descriptor 

3 Address: Port ID, Length: 64 (47 bits) 

4 See Figure 390. 
5 

6 Internal memory - Next Port 

7 Address: Port ID, Length: 64 (8 bits) 

8 See Figure 391. 
9 

10 Software Interface 

1 1 The scheduler can be programmed via the software interface by a CPU. All the 

12 scheduler's control registers and memories can also be written and read for test purposes 

13 through the software interface. The following sections list all the control registers and all 

1 4 the scheduler software interface commands. 
15 

16 Registers 

17 See Figure 392. 

18 

19 Commands 

20 Read FID NEXT Memory 

21 See Figure 393. 

22 Write FID NEXT Memory 

23 See Figure 394. 

24 Read QOS Parameters 

25 See Figure 395. 
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1 Write QOS Parameters 

2 See Figure 396. 

3 Read QOS Descriptor 

4 See Figure 397. 

5 Write QOS Descriptor 

6 See Figure 398. 

7 Read PORT Parameters 

p 8 See Figure 399. 

J; 9 Write PORT Parameters 

W io See Figure 400. 

s 11 Read PORT Descriptor 

O 
05 

f% 12 See Figure 401. 

K 13 PQ[0] = PREV_QOS_VALID 

t* 14 PQ[3:1] = PREV_QOS[2:0] 

15 PQ[6:4] = ACTIVE_PTR[2:0] 

16 

17 Write PORT Descriptor 

18 See Figure 402. 

19 Read PORT Next 

20 See Figure 403. 
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1 Write PORT Next 

2 See Figure 404. 

3 Init QOS 

4 See Figure 405. 

5 This command write the parameters to the QOS Parameter Memory and also 

6 initiates a write to the QOS Descriptor Memory with the following values: 

7 F_EMPTY = 1 

8 Rest of the fields should be reset to zero 

9 Init port 

10 See Figure 406. 

1 1 This command write the parameters to the Port Parameter Memory and also 

1 2 initiates a write to the Port Descriptor Memory with the following values: 

13 QW_EMPTY[7:0] and QA_EMPTY[7:0] = 8'bl 1 1 1_1 1 1 1 

14 Rest of the fields are reset to zero. 

1 5 Reserved Op-codesReserved Op-codesReserved Op-codes 

1 6 The op-codes 1 1 1 1 are reserved. 
17 

18 Timin g Diagrams 

19 

20 Global Timing for Input/Output Stages 

21 The input phase and the output phase should be combined during the 8 cycles 

22 budget per cell. Since we do so, there are data dependencies between input phase and 

23 output phase. Figure 407 shows the memory accesses needed for each memory for every 

24 8 cycles of input/output phase (Figure 408 shows the addresses, all data is 2 cycles later 

25 for the external memory): 
26 
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1 Input/Output stage Pipes 

2 Input Stage 

3 The input stage actually starts one cycle before cyclel by sampling a valid 

4 signal from the DBS that holds for the entire input stage cycles. If there is no valid signal 

5 asserted, the input pipe is frozen by not changing the first pipe stage (this will also save 

6 power). See Figure 408. 

7 Output Stage 

8 When working as an egress chip, the output stage starts with reading the port 

9 memory parameters using a port ID sent by the port calendar inside the DBS. When in an 

10 ingress chip mode the location to read from is determined by the INGRESS.PTR, which 

11 is a register controlled by the scheduler. 

1 2 The output stage diagram doesn't show the data dependencies to the input phase. 

1 3 Data dependencies will be discussed in the next section. See Figure 409. 
14 

15 Data Dependencies 

16 Data dependencies between the output stage and the input stage 

17 In the scheduler, the data dependencies might happen when the input stage and 

1 8 the output stage access to the same memory address. If one stage reads a memory 

19 location before the other finishes updating the same memory location, then the read result 

20 might be stale. Or if both stages write the same memory address, the later write might 

2 1 overwrite the previous write result. 

22 There is no data dependency issue for the FlowID memory since the input and 

23 output stages are mutually exclusive on a specific FlowID, i.e. the input and output stages 

24 never work on the same FlowID. If the input stage is linking a FlowID into the linked list, 

25 the output stage is not aware of the existence of the FlowID yet so there is not output 

26 phase for this FlowID. If the output stage is scheduling a FlowID, the input stage can not 

27 link the same FlowID by the principle of operation. Even if by looking at the FID Next 
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1 memory operations, there is no overlapping between the input stage and the output stage 

2 memory accesses. 

3 For the QOS memory accesses, the input stage only updates the field of F_WP at 

4 cycle 7. But the output stage may also update the F_WP field at cycle 6 of the next 8 

5 cycle slot. To resolve the data dependency issue, when the QOS address is the same for 

6 the input and the output stages, the input stage updates not only the memory field of 

7 F_WP at cycle 7 but also the register field of F_WP which stores the output stage read 

8 result of cycle3. 

9 By the same token in the port memory case, when the input and the output stages 

1 0 access to the same port memory address, the input stage updates the memory field of 

1 1 QW EMPTY at cycle 7 and at the same time it has to update the same QW EMPTY 

1 2 field of a register which stores the output stage read result at cycle 2 to avoid the data 

13 dependency. 

14 The output stage data dependencies between consecutive 8 cycle time slots 



% 1 5 The port memory accesses of the output stage take more that 8 cycles, which 

-16 implies that the port memory accesses in one time slot overlaps the port memory accesses 

Q17 in the following time slot. More specifically, before the previous 8 cycle time slot updates 
the port memory at cycle 6, the current 8 cycle slot has to read the port memory at cycle 

19 5. 

20 To avoid the data dependencies when the consecutive 8 cycle time slots access to 

2 1 the same port (there is only one port in this case), the read result has to be taken from a 

22 register which stores the write date of the previous slot memory write instead of the 

23 memory content. 
24 
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Traffic Shaoer/Meter 
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This block can operate in 2 modes: 






• Traffic ShaDer mode: in this mode the Shaper shapes specific rates to FlowIDs 

W AidAAlL/ lJIxOL/^I xxxwtw. xxx * -i * 




4 


accordine to Single or Dual Leaky Bucket algorithm. The shaper selects a Peak rate 




c 

J 


nr a Sustained rate oer FlowID depending on an accumulated credit. Once a FlowID 




6 


has been shaped, it is marked as such in a common. 
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• Traffic Meter mode: in this mode the Meter measures the rate of the incoming 




8 


FlowID traffic and mark the FlowID as a violator if the rate is above a specific 




Q 


threshold which is programmed for a FlowID. The user periodically read the 






indications in the FlowID memory. 
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4.1 Features 
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2 A FlowID can be issued from the Database at max once every 8 cycles of 
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^ a T7inwTn i c driven to the Database at max once every 8 cycles of 200MHz clock. 
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a Qh^npc innnt traffic ner FlowID using Single or Dual-Leaky-Bucket algorithm. 
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5. Shapes traffic for rates from ~48.8kbps up to 9.920Gbps. 
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a \Art<*r traffic from 1 1 8kbt>s ud to 9 920Gbps in increments of 64kbps. 
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i TTntni 094 different nrofframmable rates can be shaped simultaneously. Each 
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shaped FlowID will point to one of the entries in this table for a specific shaping 
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rate. 
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8. Up to 1024 different programmable thresholds can be metered simultaneously. 






Each metered FlowID will point to one of the entries in this table for a specific 
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Q Farh FlowID is Droerammable to have Dual Leaky Bucket, Single Leaky Bucket 




28 


or no shaping at all. 




29 


10. Can shape or meter up to 1M FlowIDs simultaneously. 




30 


1 1 . Supports microprocessor programming of the memories for FID, rates, etc. 
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1 12. 28 bits of the external memory in FID1 are used for dual leaky bucket algorithm 

2 only. It is possible not to use these bits if using single leaky bucket only. Note that 

3 for a Meter function the extra memory for the dual-leaky-bucket is not needed. 

4 See Figure 41 1 and 412. Note that the memory bits, which are not used for the single 

5 leaky bucket should be tied to zero. 

6 See Figure 413. This block doesn't need all the memories for every single mode of 

7 operation. Refer to the memory description chapter to see the usage of the memories in 

8 each mode. 
9 

10 Functional Description 

1 1 The traffic shaper receives an input phase from the Database block along with the 

12 RatelD and then starts shaping the FlowID on a per cell basis. Each time a cell is shaped 

13 the Shaper generates a message to the Database. The Database then responds with an 

14 empty indication. All packets link list and output to the PFQ are done in the Database 

15 block. 
16 

17 Leaky-Bucket Algorithm 

1 8 The basic leaky-bucket algorithm is an accumulation of credits for the given 

19 FlowID, once the amount of credit is enough for a cell, this cell is considered shaped and 

20 is transferred. The equation to calculate is: 

21 Rout = Rin * M/N 

22 When Ri„ is the input rate (based on the selected class), M input credit, N output 

23 credit. The following Figure shows an example of M/N = 2/5 : 

24 See Figure 414 

25 . 

26 Dual-Leaky-Bucket Algorithm 

27 A dual-leaky-bucket contains 2 virtual buckets per FlowID (virtual because credit 

28 is accumulated only in one bucket). One bucket shapes the traffic for a sustained data rate 

29 and the other shapes the traffic for a peak data rate. 

30 The shaped traffic starts by using the Ms/N s values of the sustained data rate. If for some 

31 reason the accumulated credits are above a specific threshold amount, then the shaped 
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traffic uses the Mp/N P values of the peak data rate until the amount of credits returns to 



2 zero. 
3 



The credit is calculated using the current time measurement. When shaping a cell, 

4 the difference between the last shaping time and the current shaping is time calculated to 

5 create AT this is the elapsed time from the last shaped cell. ACredit is calculated as 

6 follow: 

7 ACredit = AT* M s -N s 

8 New_Credit = 01d_Credit + ACredit 

If AT is long enough then a positive credit is accumulated. If AT is short then a 
negative credit is accumulated. Once New_Credit is above the threshold then the shaping 
is done with the peak rate while the credit is always calculated with the sustain data rate. 



9 
10 
11 
12 

1 3 Implementing Timing Wheel for leaky-bucket algorithm 

14 
15 



17 
18 



The timing wheel contains 256K time slots. The wheel is rotating in a rate of one 
time slot every cell cycle. Each time slot can have few FlowIDs link to it. There is one 
16 exit point in the timing wheel, which is time zero. A FlowID is linked to a future time slot 
based on its shaping parameters. Once that time slot reached time zero, all the FlowIDs 
that are linked to that specific time slot are dropped to an output FIFO. When a Flow© 

19 enters the output FIFO it means that one cell for that specific FlowID is shaped. 

20 Once every cell cycle one FlowID is taken from the output FIFO and is being serviced, 

21 means, a new time slot is calculated for the FlowID based on the ratelD and some other 

22 parameters. The FlowID then is linked to that future time slot. In parallel the shaper sends 

23 this FlowID to the Database block, in this block a cell count and packets link list are 

24 maintained per FlowID. 

25 It is possible that several FlowIDs are linked to the same time slot. When this time 

26 slot arrives to time zero, several FlowIDs are scheduled to the output FIFO at the same 

27 cell cycle. In this case the wheel services only one FlowID when the rest are waiting in an 

28 output FIFO to be serviced in the next cell cycle. See Figure 415. 

29 The timing wheel is actually fixed and only the zero reference point is moving. 

30 That means the wheel can be implemented as a memory and the zero point is a moving 

3 1 address, which is incremented once every cell cycle. 
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A single leaky bucket algorithm is implemented using the timing wheel. As described 

2 above, the Equation to implement the Leaky Bucket is: 

3 Rout = Rin*M/N 

4 When assigning K to N/M, we will get 

5 Rout = Rio I ^ 

6 The K parameter is the time in the future to place the FlowID. For example if K=3 

7 means the Ro„t is 1/3 of then the FlowID is linked to the time slot which it's distance 

8 from the current time is 3. Since the timing wheel is rotating once every cell cycle (once 

9 every 8 cycles) the next time the FlowID will be seen is after 3 cell cycles when the time 

10 slot will reach zero: 

11 See Figure 416. 
12 

1 3 Fractional value of K 

14 It is possible that K will have a fractional value and an integer value for example 

1 5 to get 40% of the bandwidth K should have the value of 2.5. It is impossible to have a 

16 fractional time slot, therefore, to place a FlowID in a future time slot only the integer part 

17 of K is used. The fractional part of K is getting accumulated and is added to the integer 

18 part only if it is greater or equal to 1. For example, if K-1.28 (78.125% of the line rate) 

19 the shaping will appear as in Figure 417. 
20 

2 1 Output FIFO in the timing wheel 

22 The output FIFO is a link list of the FlowIDs that are shaped for one cell and are 

23 waiting to be re-linked to the wheel to some future timing slot. The output FIFO is used 

24 because it is possible that more than one FlowID will be linked to a specific slot. In that 

25 case, when the slot arrives to time 0, several FlowIDs are shaped at the same time. Since 

26 the Shaper can handle only one FlowID every cell cycle, the rest of the FlowIDs have to 

27 wait. 

28 Each time a new list of FlowIDs are linked to the output FIFO, they are marked 

29 with the time arrival to the output FIFO. When calculating the future time, the result is 

30 decreased by the amount of time the FlowID was in the output FIFO. For example, 2 
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FlowlDs are linked in the output FIFO, K F1D1 =3 and K F1D2 =5. The scheduled time slots 

2 will appear as in Figure 418. 
3 

4 Dual-Leaky-Bucket using timing wheel 

5 To create a dual-leaky-bucket in timing wheel there is a need for the following 

6 parameters: 

7 • Ks for sustain data rate. 

8 • Kp for the peak data rate. 

9 • M s , M p Sustain and Peak input rates. 

10 • Threshold the threshold value of the accumulated credits to switch to peak rate. 

1 1 • Credit the total amount of positive credits (always a positive number). 

1 2 • Residue the accumulation of the fractional part of K s or Kp. 

1 3 • Time A the time of arrival to the output FIFO. 

1 4 • Time L time of last scheduled cell to a time slot. 

1 5 • Back Ground Process is used to mark timer wrapping around since measuring AT. 
| 16 There are 3 processes for a FlowID: input phase, output phase and a background process: 
^ 17 • Input phase is when the FlowID is first linked to the timing wheel, same calculation 
0 1 8 as the output phase is applied except the residue in this case is 0. 

h 19 • Output phase is when the FlowID is services in the output FIFO. 

t 20 • Background process is a round robin service between all FlowlDs (1M of them) to 

S 21 mark the wrapping around of the current time. This is done for a FlowID that 

22 becomes empty and after a while becomes non empty, the credits should be calculated 

23 properly for the FlowID. 

24 When changing the sustain rate to a peak rate then the peak rate is used until the 

25 amount of credits is down to zero. The calculation of ACredit is being done using the 

26 sustain data rate always. Figure 419 describes the use of sustain and peak rates. 
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2 Calculation of ACredit 

3 The calculation of the ACredit depends of the rates and the time elapsed from the 

4 last output phase. There is a small difference between the calculations in the output or the 

5 input phases. 

6 K is a pure number. K multiple by one time tick gives the amount of time ticks 

7 needed to shape one cell. The equation is as follow: 

8 ACredit = (AT - 1 TimeTick * K) / K = (AT - K) / K 

9 AT is the time elapsed from the last output phase. K is decremented from AT since 

10 during that time a cell was shaped. The division by K allows measuring the credits in 

1 1 terms of cells. Note that the K in the division is the pure number (which is N/M) when 

12 the K inside the parenthesis has units of time. 

1 3 • When the output phase uses the sustain or peak rates the equation to be used is: 

14 ACredit = (AT - K s ) / K s = AT/K S - 1 
• When the input phase calculates the change in the credit there is no cell shaped 

16 therefore: 

17 ACredit = AT/K S 



1 8 Equations (Algorithm) for Input phase 

19 1. NewCredit = 0 

20 2. TimeSlot = INT(Ks)// integer part of K s 

21 3. NewResidue = FRA C(K S ) // fractional part ofK s 

22 Equations (Algorithm) for Output phase 

23 1. AT = CurrentTime-Time L 

24 2. Calculate ACredit 

25 3. TempCredit = Credit + ACredit 

26 4. If(TempCredit>Threshold) 

27 NewCredit = Threshold 
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1 Select K = K P 

2 Else 

3 NewCredit = TempCredit 

4 Select K = K s 

5 5. NewTime = (INT(K + Residue) -Time A 

6 6. TimeSlot = if(NewTime<0) ? 0 else NewTime 

7 7. NewResidue = FRAC(K+Residue) 

8 8. NewMarkBits = 11 

9 Equations (Algorithm) for Background phase 

10 1. if(MarkBits==ll) NewMarkBits = 10 

11 2. else if (MarkBits== 10) NewMarkBits = 0 

12 

13 Ri„ and K factor 

14 During input phase or output phase a future time slot is calculated according to K 

15 and time variables. A slow rate, means a high K value, will cause a linking of the FlowID 

16 to a far timing slot. Whereas a high rate, means a low K value, will cause a linking of a 

17 FlowID to a closer timing slot. This chapter is to describe the exact R in for the calculation 

18 of: 

19 Rout = Rin / K 

20 The chip runs at 200MHz and the data path is 64 bits, therefore, 

21 R ^ = 200 MH* * 64hits = 12.8 Gbps 

22 There are 256K timing slots in the timing wheel, therefore, the slowest rate that it 

23 can process is: 

24 Rout(min) = 12.8G/256K = 48.828125 kbps 

25 In order to have a shaped rate of 64kbps, K should be 200,000: 

26 64kbps = 12.8G/200000 

27 That means that if a FlowID is placed in the 200000 th time slot in the future each 

28 time, its output rate is exactly 64kbps. Figure 420 shows some of the standard rates with 

29 their corresponding K. Note that the ratio between OC192 (9.92G) to 12.8G is 1.290 
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1 

2 Violator indication 

3 When using a dual-leaky-bucket algorithm the violator indication should be set 

4 when using the peak rate. When using a single-leaky-bucket algorithm, the violator 

5 indication should not be set. 
6 

7 Metering 

8 Metering is the other mode of operation of this block. The user can assign one of 

9 IK different rates pre defined by the user to a specific FlowID and the block will monitor 

1 0 the traffic to that FlowID. If the actual traffic rate of the FlowID, the measured rate, is 

1 1 greater then the assigned rate then the FlowID is marked by the hardware. 

12 The user should periodically read the marking bit per active FlowID. Once the user reads 

1 3 the FlowID information, the bit is cleared and remains clear until the input traffic is 

14 violating again. 
15 

16 External memory for Metering mode 

1 7 The meter block used the same memory bits of the FlowID but for different 

1 8 meaning. Each FlowID needs to have a counter to count number of incoming cells. Also 

19 each FlowID needs to have a bit to mark the FlowID as a violator of the assigned rate or 

20 not. 
21 

22 Background process of Metering 

23 The background process of the metering is used to check for the accumulated 

24 amount of cells per FlowID every specific amount of time. Once every 8 cycles the 

25 background process services one FlowID. That means, a FlowID is being checked by the 

26 background process once every 40mSec (at 200MHz clock). 

27 To provide more accuracy to lower rates, a counter is added to each FlowID to 

28 control the amount of times the FlowID is accessed by the background process before the 

29 actual measurement and marking is done. When the background process accesses a 

30 FlowID and this counter is greater then zero, then the background process only decrement 

3 1 that counter, when the counter is zero the background process compares the accumulated 
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amount of cells to the threshold, mark the FlowID is the threshold is smaller then the 

2 accumulated amount and then the cell counter is reset, the background counter is set to 

3 the value assign by the user. 

4 Once the metering algorithm marked a FlowID as a violator, this FlowID is to 

5 remain marked until the user reads the FlowID memory. When the user reads the FlowID 

6 memory, the mark bit is reset by the hardware. 
7 

8 Output FIFO for Metering 

9 During the input phase of a FlowID in the metering mode, the FlowID can be 
marked as a violator. If it is to be marked as a violator, then the FlowID is linked to a link 

1 1 list of violators. The software accesses this list when it wants to get the list of the 

12 violators, this way the software doesn't have to read all FlowIDs to detect the violators. 

13 Once the FlowID is linked to that list, only when the software reads it, it will be removed 

14 from the list. The software first read the value of the MARK_RP register, which is the 

15 pointer to the head of the violators link list and then read the FlowID memory. The Meter 

1 6 generates a write to the FlowID memory to clear the mark bit, 
17 

18 Block Description 

19 Shaper's Input Phase 

20 The Shaper input phase receives a ratelD, FlowID and valid signal from the 

21 DataBase block. This is a request from the database block to shape the FlowID. Once the 

22 FlowID is inside the shaper, no more input phase is initiated by the DBS block. 

23 If the specific time slot was empty then it is marked as non-empty, if the specific time 

24 slot was not empty then the FlowID is linked to the last FlowID in the link list. 
25 

26 Shaper's Slot Background process 

27 The Traffic Shaper Wheel is constantly rotating one time slot every 8 clocks of 

28 the 200MHz clock. As each time slot becomes current, the background process is 

29 activated to get al the FlowIDs from the current time slot and link them to the output 

30 FIFO link list. While linking the FlowIDs from the time slot to the output FIFO, the time 

3 1 slot is marked as empty and become the last time position in the wheel. 
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1 

2 Shaper's FlowID Background process 

3 This background process is to mark the time in each FlowID such that in the next 

4 output phase a AT is calculated correctly even if the free running time counter was 

5 wrapped around. The process resets the bits while the output phase sets the bits. Each 

6 FlowID is serviced once every 1M cell cycles (8M clock cycles). In order to mark a 

7 FlowID to use only the threshold value as a credit without using AT, this process has to 

8 be repeated 2 times. Therefore, a FlowID is marked as "too old to calculate credit" at 

9 least after 1M+1 cell cycles and no more then 2M cell cycles (16M clock cycles). In this 
1 0 case the credit value becomes the value of the threshold. 

11 

12 Shaper's Output Phase 

1 3 The output phase operates once every cell cycle on the FlowID, which is the head 

1 4 of the output FIFO's link list. Operating on a specific FlowID means shaping one cell of 

1 5 the FlowID. The shaper drives the FlowID, which is the head of the output FIFO's link 

16 list to the database block. The database block manages the link list of the packets pr 

1 7 FlowID. The act of driving a FlowID to the database block means that one cell is shaped 

18 for that FlowID. 

19 The database can return a message to the shaper that the current FlowID is empty, 

20 in this case the shaper will not link the FlowID to a future time slot. Note that the Shaper 

2 1 always shapes a cell at a time for packets traffic and for cells traffic. It is the Database 

22 block that keeps track of shaped packets/cells and generate de-queue commands to the 

23 PFQ accordingly. During metering mode, the output phase is disabled. 
24 

25 Shaper's Time measurement 

26 For the dual-leaky-bucket algorithm there is a need to measure time for 

27 calculating the change in the credit. A 2 1 bits register is used to count the time, this 

28 counter is incremented once every 8 cycles. For all calculations the logic used only 1 8 

29 bits of the time counter except for the FlowID background process. 

30 Since there are 1M FlowIDs supported there is a need to measure time up to 1M 

3 1 units and more, that' s is why the time counter has 2 1 bits. 
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2 Traffic Shap er Al gorithms 

3 The below algorithms are implementation specific and not only the mathematical 

4 equations and conditions. To understand the algorithms, one should look at the memory 

5 description first. Memory description. 
6 

7 Shaper Input Phase Algorithm (for single leaky bucket) 

8 1. Receive from database block: VALID, FID, RATEID 

9 2. RD shp_rate_id_mem[RATEID]{Ks, Kp, Threshold, M) 



10 3. RD FID2_mem[FID] {start, bg_stt, timejnsb} 

1 1 RD FIDl_mem[FID] {last time, residue, credit, p_s} 

12 4. // Calculate future time slot and residue based on previous values 

13 iff (P_s==l) & bg_stt[I:0]!=2 WO) 

14 a ) S lot_id[17:0] <= cur _slot_addr [17:0] + INT(Kp) 

15 b) n_residue[17:0] <=FRAC(Kp) 

16 C ) if(VALID & INT(Kp)==0) link_to_outJifo <= 1 else UnkJo_outJifo 

17 0 

18 else 

19 a ) s lot_id[l 7:0] <= cur _slot_addr [17:0] + INT(Ks) 

20 b) n_residue[17:0] <= FRAC(Ks) 

2 ! C ) ifrVALID & INT(Ks)==0) UnkJo_outJifo < = 1 else link_to_outJifo 

22 0 

23 5. RD slot_mem[slotJd]{rp_slot, wpjslot, ejslot) 

24 6. II Calculate new credit, see block diagram below 

25 n_credit = /(current Jime, lastjime, Ks, M, credit, threshold, bgjti) 

26 7. // Calculate rate selector for next output phase 

27 if(n_credit> ^threshold) n_p_s <= 1 

28 else if(n_credit==0) n _p_s <= 0 

29 n _p_s <=p_s 

30 5. //Create link FID to future time slot 

31 9. if (e_slot==l) 
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1 a) {n_rp_slot, n_wp_slot, n_e_slot} <= FID, FID, 0 

2 b) n start <=1 

3 10. else 

4 a) {n_rp_slot. n_wp_slot, n_e_slot} <= rp_slot, FID, 0 

5 b) n_start<=0 

6 11. if(link_to_outJifo) 

7 Link FID to output FIFO 

8 12. else if(VALID==l) 

9 WR slot_mem[slot_id]{rp_slot, wpjlot, e_slot}<= njpjlot, n_wp_slot, 

10 n_e_slot 

11 13.// Write back for FID fields 

12 if(VALID==l) 

13 FIDl_word_en[0], Wr FID l_mem[FID] {residue} <=n_residue 

14 fw l_word_en[3], Wr FIDl_mem[FID] {credit, p_s} <= n_credit, njp_s 
x 5 FID2_word_en[0J, Wr FID2_mem[FID] {start} < = n_start 

16 if(VALID==l & e_slot==0) // to avoid a write to an unknown FID 's next field 

17 FID2_word_en[l], WR FID2_mem[wp_slotJ{next_FID, bgjstt, 
lg timejnsb} <= FID, 11, cur_time_reg[20] 

1 9 Slot background Phase Algorithm 

20 1. current_slot <= current_slot + 1; //wraps around 256k 

21 2. RD shp_slot_mem[current_slot]{rp_slot, wp_slot, ejlot} 

22 3. WRshp_slot_mem[current_slot]{e_slot} <= l 'bl 

23 4. If(e_slot==l) 

24 a . do nothing //nothing to add to the output registers 

25 5. else 

26 a. wp_out_reg <= wpjslot 

27 b. wr_en_shp_FID_next_mem <= (e_out_reg==0) 

2 8 c. if(e_out_reg==0) //output fifo is not empty 

29 i. rp_out_reg <= rp_out_reg 

30 ii. ejout reg <= e_out_reg 

31 d. else 
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1 




i. rp_out_reg <= rp_slot 




2 




ii. e_out_reg <= 0 




3 


6. 


If(wr_en_shp_FIDjiextjnem ==l) 




4 




word_en[0], WR shp_FID_next_mem[wp_out_regJ{next_tlU} < rpjioi 




5 


7. 


lf(e_slot==0) 




6 




FIDljvord_en[l], WRFIDl_mem[rp_slot] {arrival Jime} <= currentjime 




7 
8 


FID Background Phase Algorithm (for dual-leaky-bucket only) 




9 


1. 


current _FID <= current _FID + i; 




10 


2. 


flZ) FID2_mem[current_FID] {empty, bgjstat, timejnsb, start} 




11 


3. 


case (bg_stat) //synopsys full case parallel case 




12 




a. 2'bll: n_bg_stat <=2'bl0 




13 




b. 2 'blO: n_bg_stat <= 2 'bOO 




14 




c. default: 2 'bOO 




15 




endcase 




16 


4. 


FID2 word_en[l], WR FID2_mem[current_FID] {empty, bg_stat, timejnsb, start}< 


: 5:5 

3=3: 


17 




n empty, n_bg_stat, timejnsb, start 




18 


Shaper Output Phase Algorithm 




19 


1. 


RD FID1 mem[rp_out_reg] {residue, arrival Jime, last time, credit, p_s} 


M= 


20 


2. 


RD FID2_mem[rp_out_reg]{next_FID, bgjstt, timejnsb, start} 


P 

: ; 


21 


3. 


Send to Database block: rp_out (as 0_FID), ~e_outjeg (as 0 VALID) 




22 


4. 


Receive RatelD from Database 




23 


5. 


RD rateid_mem[RateID]{Ks, Kp, threshold} 




24 


6. 


// Calculate future time slot and residue based on previous values 




25 




time_spent_in Jlfofl 7:0] <= current Jime[l 7:0] - arrival Jime [1 7:0] 




26 




if(p_s==l) 




27 




a) slotjd[18:0] <= cur slotjaddr [17:0] + INT(Kp) - 








time spent in Jifof 1 7: 0] 




29 




b) njesiduefl 7:0] <= FRAC(Kp) 




30 




c) iffVALID & INT(Kp)==0) linkjojjutjfo <= 1 else linkjojjutjifo <= 




31 




0 
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1 else 

2 d) slot_id[18:0] <= cur _slot_addr [17:0] + INT(Ks) - 

3 timespentin Jifo[17:0] 

4 e) n_residue[17:0] <=FRAC(Ks) 

5 a. if(VALID & INT(Ks)==0) linkJo_outJifo <= / else Unk_to_outJi/o <= 

6 0 

7 7. // Calculate new credit 

8 a. n_credit = /(current Jime, lastjime, residue, Ks, Kp, M, credit, threshold) 

9 8. 1/ Calculate rate selector /or next output phase 

1 0 i/(n_credit> threshold) n_p_s < = 1 

1 1 else i/(n_credit==0) n _p_s <= 0 

12 n _p_s <=p_s 

0 13 9. //prepare values /or FID link to a slot 

"5 14 RD slot_mem[slot_id]{rp_slot, wpjlot, ejlot) 

j 15 i/(e_slot==l) 

Uf 16 a. {n rp slot, njvp slot, n ejslot} <= FID, FID, 0 

^ 17 b. nstart <=1 

s _ 18 c. next_wr_en <= 0 

O 

3? 19 else 

Hj 20 a- {n rp sht, nwpjslot, n ejlot} <= rj?_s/of, FZD, 0 

O 21 6. n start <= 0 

22 c. next_wr_en <= 1 

23 1 0. Receive EMPTY indication from Database 

24 11.// Link FID to /uture time slot 

25 if(0_VALID==l & -EMPTY & link_to_outputJi/o) 

26 link FID to output FIFO 
21 else i/(0_VALID==l & -EMPTY) 

28 WR slot_mem[slot_id]{rp_slot, wpjlot, e_slot}<= n_rp_slot, n_wp_slot, 

29 n_e_slot 

30 12. // Write back all FID variables 

31 i/(0_VALID & -EMPTY) 
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WR FID l_mem[rp_out_reg] '{residue} <= njesidue 

2 WR FIDl_mem[rp_out_reg]{lastJime} <= current Jime_r eg 

3 WR FIDl_mem[rp_out_reg] {credit, p_s} <= n_credit, n_p_s 

4 WR FID2_mem[rp_out_reg]{bg_stt, timejnsb, start} <=U, cur_time_reg[20], 

5 nstart 

6 If(next_wr_en==l) WR FID2_mem[wp_slot]{FID_next} <= rpjmtjreg 

7 1 3- if(rp_out_reg==wp_out_reg & don 't place FID in output fifa again) e_out_reg 

8 <=1 

9 else e_out_reg < = 0 
10 14. rp_out_reg < = next FID 
11 

12 Translate {Exp, Mantissa} presentation of K to Integer and Fraction 

13 The K factors are represented in the database with mantissa and exponent. 

14 .For Ks and Kp the presentation is {1 .mantissa} x 2 exp . exp means the number of 

1 5 places to shift the decimal point to the right. 

16 . For inverse of Ks the presentation is {O.mantissa} x 2" exp . exp means the number of 
G9 1 7 places to shift the decimal point to the left. 

18 - Exp can be common for the Ks value and the InvKS value in the internal memory. 

19 The following algorithm is to translate Ks or Kp value to integer and fraction values from 

20 a presentation of {exp, mantissa} . This code is used in the input phase and in the output 

21 phase. rate_int = INT(K) and rate_frac = FRAC(K): 

22 1. case(exp) // synopsys full case parallel case 

23 0:rate_int<={17'b0,l} rate Jrac<=man(l 7:0) 

24 l:ratejnt<={16'b0,l,man(17)} rate Jrac<={man(l 6:0), l'bO) 

25 2: rate_int<={15 'b0,l,man(l 7:16)}ratejrac<={man(15:0),2 'bO} 

26 3: ratejnt<={14 'b0,l,man(l 7:15)}ratejrac<={man(14:0),3 'bO} 

27 4: rate_int<={13 'b0,l,man(l 7:14)}ratejrac<={man(13:0),4 'bO} 

28 5: rate_int<={12 'b0,l,man(l 7:13)}ratejrac<={man(l 2:0),5 W) 

2 9 6: ratejnt<={ll 'b0,l,man(l 7:12)}ratejrac<={man(l 1 :0),6'b0} 

30 7:ra teJnt<={Wb0,l,man(17:ll)}rateJrac<={man(10:0),7 , b0} 

31 8:ratejnt<={9'b0,l,man(17:10)} rateJrac<={man(9:0),8W} 
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9: rutejnt<={8'b0,l,man(17:9)} ratejrac<={man(8:0),9'b0} 

2 W:mteJnt<={7W.l,man(17:8)} ratejrac<={man(7:0),10'b0} 

3 1 1: ratejnt<={6'b0,l,man(17:7)} ratejrac<={man(6:0),ll 'bO} 

4 12: rate_int<={5'b0,l,man(17:6)} ratejrac<={man(5:0),12'b0} 
1 3: ratejnt<={4 'bO, l,man(l 7:5)} ratejrac<={man(4:0), 1 3 'bO} 

6 14: ratejnt<={3'b0,l,man(17:4)} ratejrac<={man(3:0),14'b0} 

7 15: rate_int<={2'b0,l,man(17:3)} ratejrac<={man(2:0),15 , b0} 

8 16: rate_int<={l ^0,1^(17:2)} ratejrac<={man(l:0),16'b0} 

9 default: rate_int<={l,man(17:l)} ratejrac<={man(0),17'b0} 

x 0 // make case 1 7 and above default to keep, for exp> =1 7 use 17 only 

1 endcase 

12 2. INT(K)<=rate_int-l FRAC(K) = rateJrac 

1 3 Calculation of K based on desired Rout 

1 4 Since the shaper performs the following equation: 

j g Rout = Rin I K 

1 6 K value can easily be calculated as: 

^7 K. = Rii/Rout 

1 8 For rates units of [bit/Sec] K is calculated like this: 

19 K = 12.8*10 9 /R» ut 

20 Credit calculation diagram 

21 For input phase the credit calculation depends on the following parameters: 

22 n_credit =f (current Jime, lastjime, Ks, M, credit, threshold). 

23 For output phase the credit calculation depends on the following parameters: 

24 n_credit = J '(current Jime, lastjime, residue, Ks, Kp, M, credit, threshold). 

25 Both calculations are almost the same, the implementation should be the same for both 

26 and only some inputs may change if using it for input phase or output phase. 

27 Output phase calculation is shown in Figure 42 1 . 
28 

29 Meter Al gorithms 

30 
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1 Input Algorithm 



2 /. Receive FID, RATE ID, VALID, NCELLfrom database block 

3 2. RD rateid_mem[RATEID] {threshold} 

4 3. RD FIDl_mem[FID] {cell_cnt, mark, bg_cnt, bg_cnt_value} 

5 4. n_cell_cnt_tmp[27:0] = cell_cnt[26:0] + NCELL[15:0] 

6 5. if(n_cell_cntjmp[27j==l) n_cell_cnt <= 27 "hlffffff 

7 e \ se n_cell_cnt <= n_cell_cnt_tmp[26:0] 

8 6. njnark <= mark \ (n_cell_cntJmp[27:0J >{1 'b0, threshold[26:0]}) 

9 7. n_bg_cnt <= bg_cnt 

1 0 8. if(mark=0 & n_mark==l) // this FID just violated the metering now - link it to 

1 1 violjreg 

12 if (MARK_EMPTY==1) 

13 {n_MARK_RP, n_MARK_WP, n_MARK_EMPTY} <= FID, FID, 0 

14 wr_next_FID <= 0 

15 c/je 

16 {n MARK RP, n MARK WP, n MARK EMPTY} < = MARK RP, FID, 0 

17 wr next FID <=1 

1 g e i se // either the FID did not violate metering or it is already marked 

19 {n_MARK_RP, n_MARK_WP, n_MARK_EMPTY} <= MARK RP, 

20 MARKJVP, MARK_EMPTY 

21 wr_next_FID <= 0 

22 9. if(VALID==l) 

23 WR FIDl_mem[FID] <= {n_cell_cnt, njnark, n_bg_cnt, bg_cnt_value} 

24 10. if(VALID==l & wr_next_FID==l) 

25 WR FID2_mem[MARK_ WP]{next_FID} < = FID 

26 

27 Background Algorithm 

28 I. Read FIDjnemfFID] {cell_cnt, mark, bg_cnt, bg_cnt_value} 

29 2. if(mark==l) //freeze since the FID is marked 

30 a. n_bg_cnt <= bg_cnt 

31 b. n_cell_cnt <= cell_cnt 
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1 3. else if(bg_cnt==0) If start a new counting cycle 

2 a. n_bg_cnt <= bg_cnt value 

3 b. n_cell_cnt <= 0 

4 4. else 

5 a. n_bg_cnt <= bg_cnt - 1 

6 b. n celljcnt <= cell_cnt 

7 5. Write FIDjnem[FID] <= {celljont, mark, nbg_cnt, bg_cnt_value} 
8 

9 Software read process 

10 1. RD FIDl_mem[MARK_RP] {cell_cnt, mark, bg_cnt, bg_cnt_value) 

11 2. RD FID2_mem[MARK_RPJ {next FID} 

12 3. n cell cnt = 0 

13 4. njnark = 0 

14 5. n_bg_cnt = bg_cnt_value 

15 6. if (MARK_EMPTY ==0) // some marked FIDs are avail 

16 if(MARK_RP==MARKJVP)// last FID of marked FID 's link list 
! 7 n MARK RP < = MARK UP 

j 8 n_MARK_EMPTY < = 1 

1 g/je // more than one marked FID 's avail 

20 n_MARK_RP <= nextJID 

2\ n MARK EMPTY <= 0 

22 else II n ° marked FID 's avail 

23 n_MARK_RP < = MARKJLP 

24 n_MARK_EMPTY < = 1 

25 7. if(VALID==l & MARK_EMPTY==0) 

26 FIDl_mem[MARK_RP] <= {n_cell_cnt, njnark, n_bg_cnt, 

27 bg_cnt_value} 
28 

29 Calculation of threshold for Meter mode 

30 The meter block can measure the number of cells for different intervals of times 

31 depending n the value of the BG_CNT[3:0] as shown in Figure 422, 
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1 For each measured output rate there is a maximum amount of cells that can be 

2 accumulated in 40mSec amount of time (threshold for 40mSec). The equation is: 



Threshold [#Cells/40mSec] = Rout[bits/Sec]/(25*512) = Rout/12800 

4 For example, a rate of 64,000bps means a threshold of 5 cells/40mSec. Since this number 

5 is too small, it doesn't enable much of granularity in the measurement, an increase on 

6 BG_CNT can help since the measurement will be over longer period of time. If BG_CNT 

7 is 1 5, then the threshold should increase to 5* 16 = 80 cells. 

8 The user has the tradeoff between the time to measure and the granularity of the 

9 measurement. 
10 

11 Memory description 

12 Figure 423 shows what memories and bits are needed per mode of operation. 
13 

14 Rate ID/Threshold Internal Memory 

1 5 Shaper Rate ID for Shaper mode 

16 Length: 1024, Address: 10 bit ratelD from database block, #bits: 86 

17 See Figure 424. 

1 8 Meter Threshold for Meter mode 

19 Length: 1024, Address: 10 bit ThrsID from database block, #bits: 27 

20 See Figure 425. 
21 

22 Shaper Slot Memory 

23 Length: 256k, Address: 1 8 bit slot location, #bits: 41 

24 See Figure 426. 
25 

26 Shaper/Meter FlowID Memory 

27 The FID memory is divided to 2 memory blocks with different address to each. 

28 Word write-enable signals are assigned to these memories for the shaper mode. 
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1 



Meter algorithm uses 36 bits in FID1 memory and no bits in FID2 memory. 

2 Shaper algorithm uses 65 bits in FED1 memory and 25 bits in FID2 memory. 

3 When using single-leaky-bucket only, the unused memory bits should be tied down to 



4 zero. 



6 FID MEM1 for Shaper 

7 Length: 1M, Address: 20 bit FlowID, #bits 72 

8 See Figure 427. (If more bits are needed because of future needs, the credit field 

9 can be reduced) 

10 

1 1 FID MEM1 for Meter 

12 Length: 1M, Address: 20 bit FlowID, #bits: 36 

13 See Figure 428. There is no need to word write-enable signals in case of Meter 

14 mode. 
15 

16 FID MEM2 for Shaper 

1 7 Length: 1 M, Address: 20 bit FlowID, #bits: 24 

18 See Figure 429. 

19 

20 FID MEM2 for Meter 

2 1 Length: 1M, Address: 20 bit FlowID, #bits: 20 

22 See Figure 430. 



218 



AZA-001/2001-P0I 




I CPU Interface 

2 

3 Registers Description 

4 See Figure 431. 

5 

6 CPU Commands for Shaper mode 

7 

8 Read FID1 memory 

9 See Figure 432. 
10 P - peaksustain 

I I Registers R2, R3 are used in case of dual-leaky-bucket only. 
12 

13 Write FID1 memory when using shaper 

14 See Figure 433. 

15 p - peak_sustain 

16 Registers R2, R3 are used in case of dual-leaky-bucket only. 
17 

1 8 Read FID2 Memory 

19 See Figure 434. 

20 SBT[0] - start (bit 20 in the register) 

21 SBT[2:1] - bg_stt[l:0] (bits [22:21] in the register) 

22 SBT[3] - time_msb (bit [23] in the register) 
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1 

2 Write FID2 Memory 

3 See Figure 435. 

4 SBT[0] - start (bit 20 in the register) 

5 SBT[2: 1] - bg_stt[l :0] (bits [22:21] in the register) 

6 SBT[3] - timejnsb (bit [23] in the register) 

7 

8 Read SHP_SLOT memory 

_ 9 See Figure 436. 

3 1 0 E - empty indication (bit [20] in the register) 

-5 

3 11 

w 

W 12 Write SHP_SLOT memory 

3 



O 13 See Figure 437. 

g 14 E - empty indication (bit [20] in the register) 

2 15 

16 Read RATE_ID memory 

17 See Figure 438. 

18 Ks[17:0] - Ks Mantissa, Ks[22:18] - Ks Exponent (same for Kp) 
19 

20 Write RATE ID memory 

21 See Figure 439. 
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Ks[17:0] - Ks Mantissa, Ks[22:18] - Ks Exponent (same for Kp) 



3 Setup connection command 



See Figure 440. This command resets all fields in FID1 and FID2 memories. 
Except the back_count field in FID2 memory, which is driven from the command itself. 



6 Also bg_stt[l :0] and empty bits should be set to 1 during this command. 



7 

8 CPU Commands for Meter mode 



9 

10 Read FID1 memory 

11 See Figure 441. 

12 M - Mark bit (bit [27] in the register) 

13 Cur[3:0]-bg_cnt 

14 Cnt[3:0] - bg_cnt_value 

15 

16 Write FID1 memory 

17 See Figure 442. 

18 

19 Read FID2 Memory 

20 See Figure 443. 
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1 





2 


Write FID2 Memory 




3 


See Figure 444. 




4 






5 


Read Threshold (internal) Memory 




6 


See Figure 445. 


u 


7 
8 


Write Threshold (internal) Memory 




9 


See Figure 446. 


G§ 


10 




: ; 


11 


ReauMoo write ivieier 


3 * 


12 


See Figure 447. 


M* 


13 


M - Mark bit (bit [27] in the register) 




14 


Cur[3:0]-bg_cnt 




15 


Cnt[3:0] - bg_cnt_value 




16 


This command triggers a write back to the same location clearing all values to zero 




1 7 
1 / 


fexcent the bg cnt value). That's is why this command has a different opcode then a 




18 


simple read. 




19 






20 


Setup connection command 




21 


See Figure 448. 
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1 This command writes the following data to FID1 memory: 

2 cell_cnt <= 0, mark <= 0, bg_cnt <= cnt[3:0], bg_cnt_value <= cnt[3:0] 



4 Reserved Op-codes 

5 Theop-codes 1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111 are reserved. 
6 

7 Timing Diagrams for Shaper 

8 The Shaper is able to process one input phase, one background phase and one 

9 output phase every 8 clock cycle slots. 
10 

11 Timing Diagrams for Meter 

The Meter needs to perform one input phase, one background process and one 
CPU Rd/WR accesses during 8 cycles budget. Figure 449 is a timing diagram that shows 

14 the accesses to the memories during these 8 cycles. 

1 5 The CPU RdVWr operation and the Meter software phase cannot exist together at 

16 the same 8 cycles. If a ReadModWrite command is issued then no other CPU interface is 

17 possible during these 8 cycles. 

18 For regular Read/Write from the internal/External memories, all accesses occur at 

19 slot 7. There are no data dependencies between any of the accesses. The ReadModWrite 

20 command clears all fields therefore the background process can be overwritten by it. 

21 



12 
13 
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1 MX 2 CPU Interface 



2 



This block is responsible for three different functions: 

3 . Connects between the blocks and the CPU via the CPU interface. It decodes the 

4 address to Figure which block to access and generate the chip select to each 

5 block. 

6 . Generates global synchronization signals to the rest of the blocks. 

7 . Holds general control registers like version register, mode register etc. 

8 Features 

9 • Address bus of 1 0 bits per chip . 

10 Bits 0-5 are used as an address inside the block (up to 64 registers per block) 

1 1 Bits 9-6 are used to select a block (up to 16 blocks per chip) 

12 • Data bus of 32 bits per chip. 

1 3 Externally this bus is b-directional 

14 Internally this bus is split to 2 busses: DATAJN and DATA_OUT. 

15 .Chip select signal is the qualifier for data, address and read/write busses. 

16 • The CPU interface can support multiplexed CPU interface or non-multiplexed CPU 

17 interface. When using multiplexed interface, the address and data are folded to the 

18 DATA[31:0]bus. 

1 9 • Support little endian and big endian. 

20 • CPU accesses all registers in all blocks via direct read/write commands. 

21 . CPU accesses all internal/external memories in all blocks via indirect accesses using 

22 the general-purpose registers dedicated for these accesses. 

23 • Blocks addresses are shown in Figure 450. 

24 • Generates a global sync signal once every 520 cycles of the 200MHz clock (65 ports, 

25 8 cycles per port). 

26 • Multiplexes all test busses from the blocks to one output test bus. 
27 

28 Functional Description 

29 The CPU IF block is used to interface with the CPU. The CPU IF block decodes 

30 the incoming address bus to generate a unique chip-select signal to each block. The CPU 
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IF block also holds general registers that contain general parameters and modes for the 
entire chip. Basically any logic that is not related directly to a specific block is in this 



1 

2 

3 block. 

4 The CPU IF block can operate in two modes depending on an input pins to select 

5 between them: 

6 . Regular CPU access: separate busses for data and for address. In this case address 

7 bus is qualified be the chip-select signal. 

8 . Multiplexed CPU access: one bus is multiplexed for address and for data. In this 

9 case address is qualified by the ALE signal. 

10 The CPU interface is based on Intel's microprocessor interface (when multiplexing 

1 1 address/data bus). The following timing diagrams defines the characteristics of the CPU 

12 interface for Read and for Write accesses: 
13 

14 Read access characteristics 

15 See Figure 451. 

1 6 • RDWR_L signal has the same timings as the ADDR bus. 

17 .A valid read cycle is defined when RDWR_L is latched as high on the negative edge 

18 of the CS_L signal. 

19 .In non-multiplexed address/data bus architecture, ALE should be held high so 

20 parameters t Sa ir, t Ha ir, Wl, tsir and tmr are not applicable. 

21 . In our implementation the address is latched on the negative edge of the CS_L 

22 therefore t Har is not applicable. 

23 See Figure 452. 
24 

25 Write access characteristics 

26 See Figure 453. 

27 • RDWR_L signal has the same timings as the ADDR bus. 

28 • A valid write cycle is defined when RDWR_L is latched as low on the negative edge 

29 oftheCS_L signal. 

30 • In non-multiplexed address/data bus architecture, ALE should be held high so 

3 1 parameters t Sa i w , t Ha iw, Wl, tsiw and W are not applicable. 

225 



nn . PATENT 

AZA-O01/2001-P0fl 

1 . In our implementation the address is latched on the negative edge of the CS_L 

2 therefore W is not applicable. 

3 See Figure 454. 
4 

5 Timing diagram of signals to blocks 

The following diagram describes the timings of the CPU interface to each block. 
Note that chip-select signal is now asserted for only one cycle of the 200MHz clock. For 
a write or read cycle, all parameters are valid and stable before chip-select is asserted. 
When a read cycle occurs data is driven from the block after a maximum of 3 clock 

1 0 cycles of 200MHz clock. 

11 See Figure 455. 

12 

13 Configuration modes 

14 The CPU interface has 2 configuration bits to control the CPU interface as shown 

15 in Figure 456. 
16 

17 GSYNC - Synchronization signal 

1 8 GSYNC signal is a one-cycle pulse in the 200MHz clock, which is asserted once 

19 every 520 cycles. The SYNC signal is used for synchronization between the blocks and 

20 between the chips as well as to generate the slot-count and the CPU-port indication. 

21 See Figure 457. 

22 Slot count and CPU port indication 

23 The signals slot_count and CPU _port are generated inside each block base on the 

24 GSYNC signal. Each block first needs to sample GSYNC and use the sampled version to 

25 generate the signals. See Figure 458. 
26 

27 Test Multiplexing 

28 The CPU interface block receives test busses from each block and multiplex them 

29 to one output test bus. The width of all test busses is 32 bits. Each block has a control 
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register to select the information to drive to the pins. The CPU interface block has a 

2 control register to select between the blocks. 

3 See Figure 459. 
4 

5 Interface 

The CPU IF block generates chip-select signal per block, the rdwrj, addr and 
data busses are propagated to the blocks directly. The blocks perform the read or write 
command based on the rdwrj signal only when the chip-select signal per the block is 
9 asserted. The chip-select signal per each block is synchronous to the 200MHz clock 
domain already. Each block first samples the incoming signals and then uses them. This 
is being done to solve any timing issues that may occur. If during placement and routing 
the signals from the CPU IF to the blocks will violate the 5nSec budget, another set of 

1 3 FFs can be placed in the middle of the path. 

14 For a read cycle the CPU IF block selects the data busses coming in from the blocks 

1 5 based on the chip-select per block (the address). 

16 See Figure 460; software reset and testing signals are not included in the diagram. 

17 The address, data and rd/wr signals takes about 4 cycles of 200MHz to propagate from 

18 the chip's pins to the block (6 cycles are used for delay calculation). 2 cycles are needed 

19 to perform read or write accesses in the block. 

20 Therefore a total of 8 cycles are needed to perform a read or a write access. 
21 

22 Block Descriptions 

23 

24 CPU Interface 

25 The negative edge of the CS_L signal qualifies the ADDR[9:0] and the RDWR_L 

26 signals. The positive edge of the CS_L signal qualifies the DATA bus for read and for 

27 write accesses. The CS_L is basically an enable signal to latches. Figure 461 shows the 

28 basic latching circuit and the synchronization to 200MHz clock domain: 
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Figure 462 describes the registers implemented in the CPU IF block. These 
registers act as control to the rest of the chip. The registers are implemented in the 
200MHz clock domain even if there is no need to do so since their value is fixed and 




6 


stable during normal operation. 
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Appendix A: Tags for Lookup Engine 

The following types of traffic are supported on the Maximus chip Lookup Engine 






XD IVJv/Iv. 




1 1 


1. ATM 






1 MPLS 






a. ATM 




1 A 


b. PPP 


.ft 


15 


c. Ethernet 


i 




H Frame Relay 




17 


3. Ethernet 


3 

E3 


18 
19 


4. IP Frame Relay 

(All bit numbering assumes Big Endian ordering) 


D 

J 


21 
22 
23 


See Figure 268. 

For the first version of the chip, the Flow Descriptor(FD) in Lookup Engine block 
will support Data Types 1 and 2(a) thru 2(d). Pure Ethernet, IP and Frame Relay will be 
supported in the next version of the Maximus chip. Header format references for these 




24 


traffic types are given below: 




25 






26 


HEADER FORMATS 



27 1. ATM - Tag in the Link Layer (carried in 28 bit fields - 12 bit VPI/16 bit VCI) 

28 (Computer Networks, 3 rd Edition, Andrew Tannenbaum, pg. 45 1) 
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1 See Figure 269. Each field is 8 bits wide with 12 bit VPI and 16 bit VCI field, for 

2 a total of 28 bits lookup tag. This 28 bit lookup tag is located in the 1 st 64-bit word, bits 1 

3 thru 28. 
4 

5 2(a) MPLS - ATM, 2(b) MPLS-PPP, 2(c) MPLS-Ethernet and 2(d) MPLS-FR 

6 (Tag is always between the Link and the Network Layer) 

7 (MPLS, Technology & Applications, Davie & Rekhter, pg. 1 32) 

8 The MPLS Header is inserted between the Link Layer and the Network Layer of 

9 each of the above four protocols. The detailed Link Layer description of each of the 

10 protocols is given below in other sections of this document. 

1 1 The MPLS field is the same for the four different Link Layer protocols. The exact 

12 location of this 20-bit MPLS label in the Frame varies with the Link Layer used, and is 

13 summarized in the table above. With 20 bits, the MPLS label allows having 2 20 possible 

14 tags. See Figure 270. 
15 

1 6 3. Ethernet - Tag in the Link Layer 

17 (Computer Networks, 3 rd Edition, Andrew Tannenbaum, pg. 281) 

18 Ethernet Destination Address(DA) field is used in the Lookup Engine Block. 

19 These are the 48 bits after the first 64 bits as shown below. See Figure 271 . 
20 

21 4. IP - Tag in Network Layer 

22 (Computer Networks, 3 rd Edition, Andrew Tannenbaum, pg. 413) 

23 The IP Packet Destination Address is used for Lookup Engine. Starting from the 

24 the 3 rd 64 bit word, the Destination Address is 32 bits field and is used in the Lookup 

25 Engine. This assumes Big Endian Transmission. See Figure 272. 
26 

27 5. Frame Relay (FR) - Tag in Link Layer (carried in 10 bit DLCI field) 

28 (Frame Relay Technology & Practice, Buckwalter, pg. 39) See Figure 273 . 
29 

30 6. Point to Point Protocol (PPP) 

3 1 (Computer Networks, 3 rd Edition, Andrew Tannenbaum, pg. 232) See Figure 274. 
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Appendix B: Flow Control Mechanism of CSIX-L1 

CSIX provides multiple levels of flow control and supports flow control in both 
directions. At ingress, flow control is fabric-to-TM; at egress, flow control is TM-to- 
5 fabric. 
6 

7 I ink-level Flow Control 

The link level flow control is symmetric across the CSIX interface and provides 
independent control for data and control queues. There are 2 link level queues. One queue 

10 is for control traffic and the other queue is for data traffic. For each queue there is a 

1 1 ready bit in every header indicating the congestion status for the receive queue of the 

12 respective traffic type. See Figure 275. 

^ 14 Fabric Flow Control 

The fabric flow control provides a finer level of flow control than link level flow 
control by specifying the specific TM and Class that are oversubscribed. Flow control on 
a TM port (line-end or VC) basis is the responsibility of cooperating TMs. Flow control 
frames can go in both the ingress and egress directions. See Figure 276 and 277. 
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Appendix C — MX2 Software Interface: A User Guide to Software 

21 Interface 

22 This document describes the software interface to the MX2 chip via the CPU 

23 interface. This document should include all atomic CPU commands as well as the high 

24 levl commands. 

25 There are two types of commands : 

26 1 . Low-level commands, when issued, the driver generates a single command to access 

27 registers or internal/external memories in one specific block. 

28 For example read scheduler's external memory 

29 2. High-level commands, when issued, the driver generates several commands to access 

30 registers or internal/external memories in several blocks. 
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For example setup connection command to initialize FID database in several 
2 blocks. 
3 

4 Features 



1. the interface supports up to 16 different blocks in the chip, each block has up to 64 



6 registers 
7 
8 



2. CPU accesses all registers in all blocks via direct read/write commands. 

3. CPU accesses all internal/external memories in all blocks via indirect accesses using 
9 the general-purpose registers dedicated for these accesses. 

10 4. CPU interface can support multiplexed address/data bus or non-multiplexed. 

11 5. CPU interface can support big endian or little endian modes. 
12 

13 Interface 

14 

15 Physical Interface 

The CPU interface to the MX2 chip is via standard CPU interface that contains 
signals as shown in Figure 463. The software, using this interface, can directly access the 
registers inside each block and can indirectly access internal and external memories per 
19 each block. 
20 

2 1 Address spaces per block/chip 

22 The 4 MSBs of the address bus points to the specific block, while the 6 LSBs of 

23 the address points to internal registers inside the block. Each block has address space of 

24 64 entries only. The internal registers of each block are directly mapped to these 

25 addresses and external/internal memories are indirectly mapped. The MX2 chip can 

26 support up to 16 blocks with 64 registers each. Figure 464 is the allocation of address 

27 spaces per block. 
28 

29 Direct Access 

30 A direct access is used to access registers only. The only two commands used in a 

31 direct access are read and write. Each register has it's own unique address inside the chip. 
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2 Indirect Access 

3 An indirect access is used to access internal or external memories. Each block has 

4 a set of registers dedicated for the indirect access. The COM register contains an Op- 

5 code and address of the memory to access. Registers RO through Rn are used to contain 

6 the data. 

7 Indirect Write Command 

8 A write command starts with writing the data to registers RO to Rn and then 

9 writing the opcode and the address to the COM register. The opcode defines which 

1 0 memory to access and if it is a read or write command. 

1 1 The hardware then, takes the data from registers RO through Rn and write them to a 

12 specific address defined in the COM register in the specific memory defined by the 

13 opcode. 

14 Once the command is complete, the hardware resets the opcode bits in the COM 

15 registers. It is guarantee that aread/write command to internal/external memory will be 

16 complete after 12 cycles of the operating clock (@200MHz it means 60nSec). 

17 Indirect Read Command 

18 A read command starts with writing the opcode and the address to the COM 

19 register. The opcode defines which memory to access and if it is a read or write 

20 command. 

The hardware then, uses the address to perform a read access the memory. Once 
the data arrives the hardware assign the data to registers RO through Rn and then clears 

23 the opcode bits in the COM register. 

24 The software then, generates direct reads to registers RO through Rn to get the 
data. It is guarantee that aread/write command to intemaVexternal memory will be 



21 
22 



25 



26 complete after 12 cycles of the operating clock (@200MHz it means 60nSec). 
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1 

2 CPUIF Block 

3 CPUIF block doesn't have any internal or external memories, therefore, only 

4 direct commands to registers are possible in this block. 
5 

6 Registers 

7 See Figure 465. 
8 

9 Commands 

10 N/A 
11 

12 PFO Block 

13 

14 Registers 



15 See Figure 466. 

16 

17 Commands 

1 8 Read FID Enqueue Memory (72 bits) - Up to 2M locations 

19 See Figure 467. 

20 Write FID Enqueue Memory (72 bits) - Up to 2M locations 

21 See Figure 468. 

22 Read FID Dequeue Memory (36 bits)- Up to 1M locations 

23 See Figure 469. 

24 Write FID Dequeue Memory (36 bits) - Up to 1M locations 

25 See Figure 470. 
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1 Read BID Memory (36 bits) - Up to 16M locations 

2 See Figure 471. 

3 Write BID Memory (36 bits) - Up to 1 6M locations 

4 See Figure 472. 

5 Read Stat Memory (72 bits) - Up to 2M locations 

6 See Figure 473. 

7 Write Stat Memory (72 bits) - Up to 2M locations 

8 See Figure 474. 

9 Setup connection command for FID 

10 See Figure 475 . This command sets up a connection by accessing external 

1 1 memories and initializes the corresponding locations to the proper parameters. It is a 

12 write command. Clear PFQ entries for ENQ, DEQ and STAT! 

13 Tear-Down connection command for FID 

1 4 See Figure 476. This command tears down a connection by accessing external 

1 5 memories. It is a write command. The data is all "0" PFQ returns all remaining BIDS on 

16 the queue of the down connection to the free link list during this command. 

1 7 INIT FID MEMORIES 

1 8 The memories are the enqueue, dequeue, and statistics memories. See Figure 477. 

19 INIT BID Memory 

20 This will build the free buffer list. See Figure 478. 
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1 Reserved Op-codes 

2 The op-codes 1 101, 1 1 10, and 1 1 1 1 are reserved. The following commands are 

3 combinations of the above simple commands. A user command like this should be broken 

4 to several Read or Write commands in the driver. 

5 Initialized External memories 

6 • Write "0" in all memory locations. 

7 • Write "NULL" in the enqueue memory. 

8 • Write "NULL" in the dequeue memory. 

9 • Build the free buffer list by linking each buffer to the next one. 

1 0 Initialized Internal memories 

11 • Write "NULL" in the Multicast Root memory. 

1 2 • Write "NULL" in the Multicast Leaf memory. 

1 3 • Write "NULL" in the Tunneling Root memory. 

14 • Write "NULL" in the Tunneling Leaf memory. 
15 

16 DataBase Block 

17 

18 Registers 

19 See Figure 479. 

20 

21 Commands 

22 Read external FID memory 

23 See Figure 480. 

24 A[0] - CP (bit [6] in the register) 

25 A[l] - CPU_PORT (bit [7] in the register) 

26 Pri[2:0] - Priority for shaped traffic 
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1 S - Shape (bit [23] in the register) 

2 C[0]- dpi (bit [10] in the register) 

3 C[ 1 ] - Empty (bit [ 1 1 ] in the register) 

4 Write external FID memory 

5 See Figure 481. 

6 A[0] - C_P (bit [6] in the register) 

7 A[l ] - CPUPORT (bit [6] in the register) 

8 Pri[2:0] - Priority for shaped traffic 

9 S - Shape (bit [23] in the register) 

10 C [0] - dpi (bit [ 1 0] in the register) 

11 C [ 1 ] - Empty (bit [ 1 1 ] in the register) 

12 Setup FID connection 

13 See Figure 482. 

14 A[0] - C_P (bit [6] in the register) 

15 A[l] - CPU_PORT (bit [6] in the register) 

16 Pri[2:0] - Priority for shaped traffic 

17 S - Shape (bit [23] in the register) 

1 8 During the setup connection command, the user defines the above fields only. 

1 9 The fields RP, WP, NCELL, CELL_CNT and CLP should be reset to 0 by the hardware 

20 The field EMPTY should be set to 1 by the hardware. 

21 Read external PKT memory 

22 See Figure 483. 

23 L[0] - CLP 

24 L[l] - Reserved 
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1 Write external PKT memory 



2 See Figure 484. 

3 L[0]-CLP 

4 L[l] - Reserved 

5 Read port calendar memory 

6 See Figure 485. 

7 Data[5:0]-portid 

8 Data[6] - valid 

9 Data[7]-jump 

10 Write port calendar memory 



11 See Figure 486. 

12 Data[5:0]-portid 

13 Data[6]- valid 

14 Data[7] - jump 

1 5 Read SHP Strict memory 



16 See Figure 487. 

17 P[2:0] - previous QOS pointer 

18 P[3] - previous QOS pointer valid 

19 Write SHP Strict memory 

20 See Figure 488. 

21 Read SHP out memory 

22 See Figure 489. 
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1 Write SHP out memory 

2 See Figure 490. 

3 Init SHP_out_port memory 

4 See Figure 491 . This command generates write accesses to all entries in the 

5 Shp_out memory (64 entries) with a specific data of: p_rp = 0, p_wp - 0, p_e = 1 

6 Init Shp_strict memory 

7 See Figure 492. This command generates write accesses to all entries in the 

8 Shp_strict memory (64 entries) with a specific data of: empty = 8'bl 1 1 1_1 1 1 1 , 
^ 9 prev qos v - 0, prev qos = 0 

SI 10 Reserved Opcodes 

Ui 1 1 The opcodes: 1 1 1 0, 1 1 1 1 are reserved. 
2 12 

s 13 Shaper/Meter Block 

| 14 

f=f 15 Registers 

O 16 See Figure 493. 
f " 17 

1 8 Commands for Shaper mode 

19 Read FID1 memory 

20 See Figure 494. 

21 P - peak_sustain 

22 Registers R2, R3 are used in case of dual-leaky-bucket only. 
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1 Write FID1 memory when using shaper 



2 See Figure 495. 

3 P - peak_sustain 

4 Registers R2, R3 are used in case of dual-leaky-bucket only. 

5 Read FID2 Memory 

6 See Figure 496. 

7 SBT[0] - start (bit 20 in the register) 

8 SBT[2:1] - bg_stt[l :0] (bits [22:21] in the register) 

9 SBT[3] - time_msb (bit [23] in the register) 

p 10 Write FID2 Memory 

iff. 

11 See Figure 497. 

$ 1 2 SBT[0] - start (bit 20 in the register) 

% 13 SBT[2:1] - bg_stt[l :0] (bits [22:21] in the register) 

14 SBT[3] - time_msb (bit [23] in the register) 

3 

J 15 Read SHP_SLOT memory 

□ 16 See Figure 498. 

^17 E - empty indication (bit [20] in the register) 

1 8 Write SHP_SLOT memory 

19 See Figure 499. 

20 E - empty indication (bit [20] in the register) 

21 Read RATE ID memory 

22 See Figure 500. 

23 Ks[17:0] - Ks Mantissa, Ks[22:18] - Ks Exponent (same for Kp) 
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1 Write RATE ID memory 



2 See Figure 501. 

3 Ks[17:0] - Ks Mantissa, Ks[22:18] - Ks Exponent (same for Kp) 



4 Setup connection command 



5 



See Figure 502. 



6 This command resets all fields in FID1 and FID2 memories. Except the back_count field 

7 in FID2 memory, which is driven from the command itself. 

8 Also bg_stt[l :0] and empty bits should be set to 1 during this command. 
9 

1 0 Commands for Meter mode 

1 1 Read FID1 memory 

12 See Figure 503. 

13 M - Mark bit (bit [27] in the register) 

14 Cur[3:0]-bg_cnt 

1 5 Cnt[3 :0] - bg_cnt_value 

16 Write FID1 memory 

17 See Figure 504. 

18 Read FID2 Memory 

19 See Figure 505. 

20 Write FID2 Memory 

21 See Figure 506. 
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1 Read Threshold (internal) Memory 



2 See Figure 507. 

3 Write Threshold (internal) Memory 

4 See Figure 508. 

5 ReadModWrite Meter 

6 See Figure 509. 

7 M - Mark bit (bit [27] in the register) 

8 Cur[3:0]-bg_cnt 

9 Cnt[3 : 0] - bg_cnt_value 

10 This command triggers a write back to the same location clearing all values to 

1 1 zero (except the bg_cnt_value). That' s is why this command has a different opcode then a 

12 simple read. 

13 Setup connection command 

14 See Figure 510. This command writes the following data to FID1 memory: 

1 5 cell_cnt <= 0, mark <= 0, bg_cnt <= cnt[3 :0], bg_cnt_value <= cnt[3 :0] 

16 Reserved Op-codes 

17 The op-codes 1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111 are reserved. 
18 

19 Scheduler Block 

20 

21 Registers 

22 See Figure 511. 
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1 Commands 

2 Read FID NEXT Memory 

3 See Figure 512. 

4 Write FID NEXT Memory 

5 See Figure 513. 

6 Read QOS Parameters 

7 See Figure 514. 

8 Write QOS Parameters 
^ 9 See Figure 515. 
W 10 Read QOS Descriptor 

i - 

-St: 

11 See Figure 516. 

12 Write QOS Descriptor 

13 See Figure 517. 

14 Read PORT Parameters 

15 See Figure 518. 

16 Write PORT Parameters 

17 See Figure 519. 

1 8 Read PORT Descriptor 

19 See Figure 520. 



5 

P 
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1 PQ[0] = PREV QOS VALID 

2 PQ[3:1] = PREV_QOS[2:0] 

3 PQ[6:4] = ACTIVE_PTR[2:0] 
4 

5 Write PORT Descriptor 

6 See Figure 521. 

7 Read PORT Next 

8 See Figure 522. 

9 Write PORT Next 

D 
\G 

S3 10 See Figure 523. 

'ff s 11 InitQOS 

M° 12 See Figure 524. This command write the parameters to the QOS Parameter 

□ 1 3 Memory and also initiates a write to the QOS Descriptor Memory with the following 
^ 14 values: 
M 15 F EMPTY = 1 

5 1 6 Rest of the fields should be reset to zero 



17 Initport 

1 8 See Figure 525. This command write the parameters to the Port Parameter 

19 Memory and also initiates a write to the Port Descriptor Memory with the following 

20 values: 

21 QW_EMPTY[7:0] and QA_EMPTY[7:0] = 8'bl 1 1 1_1 1 1 1 

22 Rest of the fields are reset to zero. 
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1 Reserved Op-codesReserved Op-codesReserved Op-codes 

2 The op-codes 1 1 1 1 are reserved. 
3 

4 Driver's high level commands 

5 The following section describes higher level commands then just read/write to 

6 internal/external memories. The following commands are basically combinations of the 

7 above commands and are created to give easier user interface to the chip. 

8 These commands doesn't have special opcode in the chip itself, they are implemented in 

9 the driver only. 
10 



1 1 Setup Connection Command 

*== 12 This command sets up a connection by accessing external memories in several 

* 13 blocks and initializes the proper parameters per FlowID. 

^ 14 The following table describes the parameters per FlowID that should be set during 

S 1 5 the setup connection command, for more specific information about the fields, please 

W 1 6 refer to the blocks SPECs in Figure 526. 

s 1 7 Once the driver receives a setup connection command, the driver generates setup 



fe; 1 8 connection command to the DBS, SHP and PFQ blocks. The parameters to the setup 

O 1 9 connection commands to the blocks are driven from the user along with the global setup 

hs 20 connection command to the driver. 

TEST 

H 21 

22 Tear-down Connection Command 

23 A tear-down-connection command is used only in the PFQ block. This command 

24 is needed to invalidate some fields in the FID descriptor memories as well as the statistics 

25 fields. The Database, Shaper and Scheduler blocks don't need to invalidate any of the 

26 FID fields. Therefore, there is no special driver command for this command. The user can 

27 initiate a simple write command to the PFQ to implement this command. 
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1 

2 Power-up initialization 

3 When reset is applied to the chip (or power-up) the internal control registers wakes up 

4 in their default value, all traffic is disabled and internal/external memories are not 
defined. The following is the initialization sequence the user should perform before 



5 



6 allowing any traffic to the MX2 chip: 



7 



8 1. PFQ - issue "Init FID Memory" command 

9 2. PFQ - issue "Init BID Memory" command 

10 3 . PFQ - initialize multicast internal memories (if needed) 

11 4. PFQ - initialize tunneling internal memory (if needed) 

12 5 . PFQ - specify bits in control register if different then default (address 3 6) 
5 13 6. PFQ -initialize RED internal memory only if using RED 

S3 14 7. PFQ - initialize class threshold registers (addresses 45 to 52) only when using 

J 15 CLASS 

W 16 8. DBS - initialize Port Calendar memory 

j=I 17 9. DBS - issue "Init Shp_strict Memory" command 

L 18 1 0. DBS - issue "Init Shp_out_port Memory" command 

IP 19 11. CPUIF - define Shaper/Meter mode (register 1 bit [8]) 

U 20 12. SHP - initialize RatelD/Threshold memory 

O 21 13. SHP - issue "Init Slot Memory" command 

22 1 4. SCH - initialize Port Descriptor Memory 

23 15. SCH - initialize QOS descriptor Memory 

24 16. <setup connections if needed> 

25 1 7. PFQ - enable input traffic for specific ports by writing to registers 43 and 44 

26 18. SCH - enable output traffic from scheduler (register 32 bit [0]) 

27 19. SHP - enable output traffic from shaper (register 32 bit [ 1]) 
28 

29 Once these steps are complete, the user can initiate setup connection commands. 
30 



245 



AZA-OOl/2001-POi 



PATENT 



1 The user doesn't have to go through all of the above steps, for example, if tunneling is 

2 not used then there is no need to initialize the internal memory for tunneling in the PFQ 

3 block.the MX2 Chip 
4 

5 Appendix D — MX2 External Memory 

6 See Figure 527. 
7 

8 While this invention has been described in terms of several preferred embodiments, 

9 there are alterations, permutations, and equivalents, which fall within the scope of this 

10 invention. It should also be noted that there are many alternative ways of implementing 

1 1 the methods and apparatuses of the present invention. For example, the present invention 
O 1 2 may be practiced by integrating line card functionality with switch fabric functionality 

3 1 3 such that a router does not have individual line cards. Alternatively, router in accordance 

li 14 with the invention may include many lines cards deploying the Maximus device, some of 

if! 

5 1 5 the line cards functioning as ingress line cards, others of the line cards functioning as 

5 16 egress line cards. It is therefore intended that the following appended claims be 

L 17 interpreted as including all such alteration, permutations, and equivalents as fall within 

0s 1 8 the true spirit and scope of the present invention. 

J s 
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